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(57) Abstract: Interactive, computer based orthodontist 
treatment planning, appliance design and appliance 
manufacturing is described. A scanner is described which 
acquires images of the dentition which are converted to 
three-dimensional frames of data. The data from the 
several frames are registered to each other to provide a 
complete three-dimensional virtual model of the dentition. 
Individual tooth objects are obtained from the virtual 
model. A computer-interactive software program provides 
for treatment planning, diagnosis and appliance from the 
virtual tooth models. A desired occlusion for the patient is 
obtained from the treatment planning software. The virtual 
model of the desired occlusion and the virtual model of the 
original dentition provide a base of information for custom 
manufacture of an orthodontic appliance. A variety ot 
possible appliance and appliance manufacturing systems 
are contemplated, including customized archwires and 
customized devices for placement of off-the shelf brackets 
on the archwires, and removable orthodontic appliances. 
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BACKGROUND OF THE INVENTION 

A. Field of th e Invention 

This invention relates generally to the field of orthodontics. More particularly, the 
invention relates to a computerized, interactive method and associated system for 
Orthodontic treatment. The system includes a hand-held optical seamier capturing 3- 
dimensional information of objects, interactive computer-based treatment plamiing using 
three-dimensional tooth objects and user specified simulation of tooth movement, and 
appHance manufacturing apparatus, including bending machines. 

B. Descriptinn of TRelfltftH. Ah- 

IQ orthodontics, a patient suffering from a malocclusion is typically • treated by 
bonding brackets to the surface of the patient's teeth. The brackets have slots for receiving 
an archwire. The braclcet-archwire interaction governs forces applied to the teeth and 
defines flie desired^direction of to^pth m^^^^ 

manually by the orthodontist. During the course of treatment, the movement of the teeth is 
momtored. Corrections to the bracket position and/or wire shape are made manuaUy by the 
orthodontist. 
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The key to efficiency in treatment and maximum quality in results is a realistic 
simulation of the treatment process. Today's orthodontists have the possibility of taking- 
plaster models of the upper and lower jaw, cutting the model into single tooth models and 
sticking these tooth models into a wax bed, lining them up in the desired position, the so- 
5 called set-up. This approach allows for reaching a perfect occlusion without any guessing. 
The next step is to bond a bracket at every tooth model. This would tell the orthodontist the 
geometry of the wire to run through the bracket slots to receive exactly this result. The next 
step involves the transfer of the bracket position to the origiual malocclusion model. To 
make sure that the brackets will be bonded at exactly this position at the real patient's teeth, 
10 small templates for every tooth would have to be fabricated that fit over the bracket and a 
relevant part of the tooth and allow for reliable placement of the bracket on the patient's 
teeth. To increase efficiency of the bonding process, another option would be to place each 
siagle bracket onto a model of the malocclusion and then fabricate one single transfer tray 
per jaw that covers all brackets and relevant portions of every tooth. Using such a transfCT 
15 tray guarantees a very quick and yet precise bonding using indirect bonding. 

However,-it-is-obvious-that such an approach requires-an-extreme-amount pf^ 

and labor and thus is too costly, and this is the reason why it is not practiced widely. The 
normal orthodontist does not fabricate set-ups; he places the brackets directly on the 
patient's teeth to the best of his knowledge, uses an off-the-shelf wire and hopes for the 
20 best. There is no way to confirm whether the brackets are placed correctly; and 
misplacement of the bracket will change the direction and/or magnitude of the forces 
imparted on the teeth. While at the beginning of treatment things generally run well as all 
teeth start to move at least into the right direction, at the end of treatment a lot of time is lost 
by adaptations and corrections required due to the fact that the end result has not been 
25 properly planned at any point of time. For the orthodontist this is still preferable over the 
lab process described above, as the efforts for the lab process would still exceed the efforts 
that he has to put in during treatment. And the patient has no choice and does not know that 
treatment time could be significantly reduced if proper planning was done. 
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U.S. Patent 5 437 ^ic-) ^ ^ . PCT/usoi/n969 

approach to orthodontics. In this m th d. ' ^ ^computerized, appliance-driven 

acquired. A uniplanar target archfoiBT' 1 ''""'^ i^ormation of teeth is 

of custonrized bracket slots, the bracJcetHl ^""^ '^'^^ information. The shape 

5 calculated in accordance with a rr..J '^^^^ orthodontic archwire, are 
A^cireiko et al. method is t! ^"7"^^" "^'^ ^^^^-^^ 

o-odontics b. replacing the hu.: : T^^' ^^^^"^ 

detenninistic, mathematical comnnt^t," . orthodontic appliance design with a 

'5.2 patent teaches - avlrCj. ^ -^^^^^^ Hence 
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More recently, in the late Z990's Ah>n T ^ , 
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Plaster model of the dentition of the X^PT'^'' ^ ^^^^ ^ 

-mote appliance manufacturing center, wherl itT ^"^^^^^ *° ^ 
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center and made available for\iewinJT ^^^"^^ ^PPKiS^^ manufacturing 

Orthodontist indicates changes d.ey ^sh J.^, ^f?" ^^-t. The 

another virtual nxodel is provided over the I Z ^^^"^ ^^^^'^^^^ 
model, and indicates any further changes orthodontist reviews the revised 

is agreed upon. A series of removL' ^""^ ^^^-^^ -^ti- 

dehvered to the orthodontist The shel, ""^^ ^"""^ manufactured and 

desiredortargetposition Patient's teeth to the 
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The art has lacked an effective, computer-based interactive orthodontic treatment 
planmng system that provides the necessary tools to allow the orthodontist to quickly and 
efficiently design a treatment plan for a patient. The art has also lacked a treatment 
planning system in which the orthodontist-derived parameters for the treatment can be 

5 translated into a design of an orthodontic apphance m real time, while the patient is in the 
chair. Real-time apphance design as described herein also allows for real-time 
communication of the treatment plan or apphance design to occur with the patient, or 
transmitted over a communications hnk and shared with a coUeague or remote apphance 
manufacturing faciUty. Alternatively, the treatment planning can be performed remotely 

10 and a digital treatment plan sent to the orthodontist for review, interactive modification, or 
approval. 

Scanners are devices for capturing and recording mformation firom a surface of an 
object. Scanners for obtammg mformation from a two-dimensional surface, such as readhig 
bar codes or characters printed on a piece of paper, are widely known. Several scanners 
15 have been proposed for recording three-dimensional mformation as well, mcluding the field 



" ofdehfisey. 

U.S. Patent 4,837,732 and U.S. Patent 4,575,805 to Brandestini and Moermann 
propose a scanning system for in vh'o, non-contact scanning of teeth. The patents describe a 
procedure for optically mapping a prepared tooth with a non-contact scan-head. The scan- 
20 head deUvers the contour data, converted to electrical format, to be stored m a memory. A 
computer reads the memory following a hne scan pattern. A milling device is slaved to 
follow this pattern by means of position control signals and mills an implant for the 

prepared tooth cavity. 

The scan-head of the '732 and '805 patents mcludes a Ught emitting diode, with 

25 mtegral lens that radiates Ught onto the cavity. Before reaching the object, the rays of light 
are reflected by a mirror and pass through a ruhng consisting of a plurahty of parallel slits, 
or an altematmg pattern of parallel opaque and transparent stripes. The reflected hght is 
focused by a lens onto a charge-coupled device (CCD) sebsbr. Depth information is 
determined in accordance with a principle known as "active triangulation," using parameters 

30 shown in Figure 9 of this document and described subsequently. Basically, the object is 
viewed under an angle different firom the mcident rays due to a parallax effect. Each light 
stripe will have an apparent positional shift and the amount of the shift at each pomt along 
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each Ught stripe is proportional to the vertical height of the corresponding portion of the 
surface on the object. 

U.S. Patent No. 5,372,502 to Massen et al. describes an optical probe for measuring 
teeth that works on a similar principle. As noted in the Massen et al. patent, the Brandestini 
et al. technique is difBcult to use when there are large variations in surface topography 
since such large jumps displace the pattern by an amount larger than the phase constant of 
the pattern, making it difficult to reconstmct the pattern of lines. Furthermore, precise 
knowledge of the angle of incidence and angle of reflection, and the separation distance 
between the light source and the detector, are needed to make accurate determinations of 
depth. Furthermore, the scanner has to be rather qarefully positioned with respect to the 
tooth^and would be unable to make a complete model of the dentition. 

U.S. Patent Nos. 5,027,281 to Rekow et al. describes a scanning method using a 
three axis . positioning head with a laser source and detector, a rotational stage and a 
computer controUer. The computer controller positions both the rotational stage and the 
positionmg head. An object is placed on the rotational stage and the laser beam reflects 
i*- reflected laser beam is used to measure the distance between tre-object-and the 
laser source. X and Y coordinates are obtained by movement of the rotational stage or the 
positioning head. A three-dimensional vktual model of the object is created from the laser 
scanning. The '281 patent describes using this scanning method for scanning a plaster 
model of teefli for purposes of acquiring shape of the teeth to form a dental prosthesis. The 
system of the '281 patent is not particularly flexible, since it requires the object to be placed 
on the rotational stage and precise control of the relative position of the object and tiie 
positioning head is required at aU times. Itis unsuited for in vivo scanning of the teeth. 

U.S. Patent 5,431,562 to Andreiko et al. describes a method of acquiring certain 
shape information of teeth from a plaster model of the teeth. The plaster model is placed 
on a table and a picture is taken of the teeth using a vide6 camera positioned a known 
distance away from the model, looking directly down on the model. The image is 
displayed on an input computer and a positioning grid is placed over the image of the teeth. 
The operator manuaUy inputs X and Y coordinate information of selected points on the 
teeth, such as the mesial and distal contact points of the teeth. An alternative embodiment is 
described in which a laser directs a laser beam onto a model of the teefli and the reflected 
beam is detected by a sensor. The patait asserts.tiiat three-dimensional infonnation as to 
teeth can be acquired from this technique but does not explain how it would be done. 
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Neither of the techniques of Andreiko have met with widespread commercial success or 
acceptance iq orthodontics. Neither technique achieves in vivo scanning of teeth. 
Moreover, the video technique does not produce complete three-dimensional information as 
to the teeth, but rather a limited amount of two-dimensional ioformation, requiring 
5 significant manual operator input. Even using this technique, additional equipment is 
required even to describe the labial surface of a tooth along a single plane. 

The art has also lacked a reliable, accurate, low-cost, and easily used scanning 
system that can quickly and automatically acquire three-dimensional information of an 
object, without requiring substantial operator input, and in particular one that can be held in 
10 the hand and used for in vivo scanning or scanning a model. The present iavention meets 
this need. 
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workstation. The software further includes navigation tools, e.g., typed connnands, icons 
and/or graphical devices superimposed on the displayed model, that enables a user to 
manipulate the model on the display and simulate the movement of at least one tooth in the 
model relative to oflier teeth in the model m three-dimensional space, and quantify the 
5 amount of movement precisely. This simulation can be used, for example, to design a 
particular target situation for the patient. 

The development of a unique target situation for the patient has utility in a variety of 
different orthodontic appliances, including an approach based on off-the-shelf or generic 
brackets and a custom orthodontic archwire. The scope of the invention is sufficient to 
10 encompass other types of appliances, such as an approach based on customized brackets, 
retainers, or ranovable aligning devices. In a bracket embodiment, the memory contains a 
library of virtual, three-dimensional orthodontic brackets. The software permits a usa: to 
access the virtual brackets through a suitable screen display, and place the virtual brackets 
on the virtual model of the dentition of the patient. This bracket bonding position can be 
15 customized on a tooth by tooth basis to suit individual patient anatomy. Because the tooth 
_..models,_brackets_and archwire a re individual ob j ects, and stored as such in me mory, the 
treatment planning apparatos can simultaneously display the virtual brackets, the archwire 
and the virtual model of the dentition, or some lesser combination, such as just the brackets, 
just the dentition, or the brackets and the archwire but not the teeth. The same holds 

20 trae with other appliance systems. 

In a preferred embodiment, the virtual model of teeth comprises a set of virtual, 
individual three-dimensional tooth objects. A metiiod of obtaining the tooth objects from a 
scan of teeth, and obtainmg other virtual objects of associated anatomical structures, e.g., 
gums, roots and bone is described. When the teeth are separated from each other and from 

25 the gums, they can be individually manipulated. Thus, the individual tooth objects can be 
individually selected and inoved relative to other teeth in the set of vhtual tooth objects. 
This feature peraiits individual, customized tooth positioning on a tooth by tooth basis. 
These positioning can be in terms or angular rotation about three axis, or- translation m 
transverse, sagittal or coronal planes. Additionally, various measurement features are 

30 provided for quantifying the amount of movement. 

One of the primary tools in the treatment planning apparatus is the selection and 
customization or a desired or target archform. Again, because the teeth are individual tooth 
objects, they can be moved independentiy of each other to define an ideal arch. This 
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development of the target archfonn could be calculated using mteipolation or cubic spline 
algorithms. Alternatively, it can be customized by the user specifying a type of archform 
(e.g. Roth), and the tooth are moved onto that archfonn or some modification of that 
archfonn. The archfonn can be shaped to meet the anatomical constraints of the patient. 
After the initial archfonn is designed, the user can again position the teelh on the archfonn 
as they deem appropriate on a tooth by toofli basis. The treatment planning software thus 
enables the movement of the virtual tooth objects onto an archfonn which may represent, at 
least in part, a proposed treatment objective for the patient. 

Numerous other features are possible with the treatment planning software, 
mcluding movement of the teeth with respect to the other teeth m the archfonn, changing 
the position of the virtual brackets and the teeth with respect to each other, or opposing teeth 
with respect ot the selected archfonn. Custom archwire bends can be simulated to provide 
additional corrections. Bonding conections at the bracket-tooth interface are also possible. 

In another aspect of the mvention, a method is provided for digital treatment 
planning for an orthodontic patient on a workstation having a processing unit, a user 
mteiface including a 

comprises the steps of obtaining and storing a three-dimensional virtual model of teeth 
representing the dentition of the patient in a cmrent or observed situation. The virtual model 
IS displayed on the display. The method further includes the step of moving the position of 
teem in the virtual model relative to each other so as to place the teeth of the virtual model 
into a target situation and displaying the virtual model with the teeth moved to the target 
situation to the user. Parameters for an orthodontic apphance to move the patient's teeth 
from the cuir^it situation to Ae target situation can be derived from the virtual model and 
the target situation. For example, if virtual brackets are placed on the teeth, their location 
in the target situation can dictate the design of an archwire to move the teeth to the target 
situation. 

The scanner system is provided for capturing three-dimensional infonnation of an 
object. The object can be vutually any object under scmtiny, however the present document 
will describe an application in which the object is .die dentition of a patient suffering from a 
malocclusion. 

The scanning system enables three-dimensional surface infonnation to be obtained 
with a v«y high decree of precision. Moreover, the scanning system can be used without 
requiringprecise movement of the scanner, or requiring the object under scmtiny to be fixed 
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in space. Surprisingly, the scanner is able to generate precise three dimensional surface 
information by simply moving the seamier over the surface of the object, such as by hand, m 
any manner that is convenient for the user, even if the object moves in any random direction 
during the scanning within reasonable limits. Thus, the scanner can be used to capture die 
surface of a patient's dentition in a minute or two, even if the patient moves their head or 
jaw while the scanning is occurring. Precise knowledge of the spatial relationship between 
the scanner and the object is not required. 

The scanner obtains a set of images, which are processed in a computer to calculate 
the surface configuration of the object in three dimensions of space automatically, quickly, 
with high precision, and with essentially no human involvement other than the act of 
scanning. The precision or accuracy will be dictated largely by the extent to which the 
object under scrutiny tends to have undercut or shadowed features which are difficult to 
detect, necessitating a narrow angle between the projection and imaging axes. For teeth, an 
accuracy of under 20 or 30 microns is possible. This accuracy can be further improved 
depending on the nature of the surface, such as if the surface does not have a lot of undercut 
-or shadowed-features, -by-increasing-the-angular_ separation.of_the„proj_ecjd^^ . . 

imaging axis. 

Each image captured by the scanner is converted to a virtual, three-dimensional 
point cloud or "fi-ame." The illustrated embodiment has a relatively coarse resolution for 
any single frame, due to a coarse projection pattern, but a fine resolution is obtained by 
obtaining multiple images and performing a registration procedure on the firames, as 
described below. Since each point on the surface of the object is captured m a plurality of 
images (such as five or six in a typical example of scanning), the registration of firames 
results in a fine resolution. An even finer resolution can be obtained by scamiing slower and 
capturing more images of the surface of the object from different perspectives and 
registering the resulting frames to each other. 

This surface configuration of the object in three dimensions of space can be 
represented as a mathematical model, i.e., a virtual model of the object, which can be 
displayed on any workstation or computer using available software tools. The 
mathematical model can be viewed in any orientation in space, permitting detailed analysis 
of the surface. The model can be compared to template objects stored in a computer. 
Deviations in the object from die template can be quantified and analyzed. Further, the 
virtual model can be transported from one computer and another computer anywhere in the 

11 



20 



25 



30 



wo 01/80761 

, , . PCT/US01/liy69 

such that it can be k m . ^"^^^o^. scanner can be scaled down in size 

objec. such a. l^^^LT, T "^"^^"■"^^'-^'-^--'^ 
works of art, sculptures, archeological sites /e a ti,« 

cau. 4e pn^jeotion pa«™ ^ become ^^1^; I T™ """^ ""^^ ^ 

b. i=«^ ..woe . t.o^^\„^i"n:: : '° " ^ """^ 

arc proceed to 3c«,^ ^ d^l ! "'""^ 

The seaming system, in a prefe„ed embodiment further incI„H 
stores data representing a aree axis V 7. , ""^'"^ » """"ory that 

caHhta^o^reMot.hip^r^rthf.olT^™'^"'''"*^ 

taction (e g polvno,^., °° ""'^ *™ ^ "^l^ in tbe &na of a math«taatical 

numerous portions of the pa,t«n said nix,, . ''^ 

i..o„natio„^m.heproJec.i„:sZr Z^r^T ^ ^"^^ 
and (2) distance infonnation in X a,«, y^."^™ ^' ^' <Mff-nt 2 distances, 
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pattern, at the at least two different Z distances. A method of obtaining the calibration 
relationship is described in detail below. While the simplest form of the relationship is a 
table, as described in detail below, these calibration relationships could be equivalently 
represented by one or more mathematical functions as wiU be apparent to those skilled in 

5 art. , 

The calibration relationship is used to derive three-dimensional coordinates for 
points on the surface of an object imaged by the imaging device. The generation and use 
of the preferred calibration table is also explained in further detail below. Other caUbration 
tables or procedures are also possible. The use of the caUbration relationship allows the 
10 scanner to operate without precise knowledge of the optical and mechanical properties of 
the scanner, as is required in prior art sj^tems. 

The scanning system further includes at least one data processing unit, e.g., the 
central processing unit of a computer or a digital signal processor, which processes the 
images of the projection pattern after reflection from the surface of the object. The 
15 processing unit compares data from the image (e.g., pixel locations where certain points m 
■ ■- the projection patt<^m arp. imaged) to the calibration relationship to thereby d erive spatia l 
information, in three dimensions, of points on the object reflecting the projected pattern 
onto the electronic imaging device. Mioltiple processing units can be used to reduce the 
amount of time it takes to process the two-dimensional images, calculate three-dimensional 
20 coordinates for points in each image, and register frames of three-dimensional coordinates 
relative to each other to generate a complete virtual model of the object. 

The scanning system thus dCTives three-dimeaisional information of the object from 
the images generated by the imaging device and from the calibration relationship stored m 
memory. Precise knowledge of the optical characteristics of the scanner, the angles between 
25 the first and second optical axes, the angle between the optical axis and the point in the 
imaging device, or the separation distance of the projection device from the imaging device 
are not necessary. The projection system and the imaging device can be even uncoupled 
relative to each other. The calibration relationship automatically and completely 
compensates for these issues, as well as manufacturing variations and tolerances in the 
30 scanner optics, in a highly reliable manner. Moreover, knowledge of the distance from the 
scanner to the object is not required. Additionally, control over the distance between the 
seamier and the object is not required, within reasonable limits dictated by the depth of 
focus of the imaging optics. Ideally, during scanning the distance from the scaimer to the 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is an illustration of an orthodontic care system incorporating a hand-held 
scanner system in accordance with a representative embodiment of the invention. The 
hand-held scanner is used by the orthodontist or the assistant to acquire three-dimensional 
information of the dentition and associated anatomical structures of a patient and provide a 
base of information to diagnose and plan treatment for the patient. 

Figure 2 is a block-diagram of a scanning system, suitable for use in the orthodontic 
care system of Figure 1. 

Figure 3 is a perspective view of a hand-held scanner used to acquire information of 
an object under scrutiny, suitable for use in the orthodontic care system of Figure 1. 

Figure 4 is an illustration of a patient being scanned with the hand-held scanner of 
Figure 3. 

Figure 5 is a block diagram of the back office server of Figure 1 showing the 
elements used to calculate the digital model of the patient's dentition and display the digital 
model on a screen display of the server. 

Figure -6-is-a.flow_diagram. illusn:ating. the seq uence o f ste ps used by the processing 

unit in the scanning station to calculate three-dimensional information of an object from 
images captured by the scanner. 

Figure 7 is a cross-sectional view of the scanner of Figure 3, showing the optical 
elements of the projection and imaging aspects of the scanner. 

Figure S is a perspective view of a scanner calibration apparatus of station that is 
used to calibrate the scanner and obtain data for entry in a calibration table stored in the 
memory of the scamer system. 

Figure 8A is a perspective view of an altanative embodiment of a calibration device 
for the scanner in wHch the Z direction and X-Y caHbration surfaces of Figure 8 are 
combined into a single surfece. 

Figure 9 is an illustiration of the relevant parameters that can be used to calculate 

surface configuration ofthe object in accordance with a known fashion. 

Figure 9A is an illustration of an electironic imaging device of a scanner, associated 
imaging lens system, and an object reflecting a projection pattern onto tiie imaging device at 
two different distances. 
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Figure 20 illustrates the points of the X-Y caUbration plane from the calibration 
station of Figure 7 at two positions relative to the optical elements of the electronic imaging 
device. 

Figure 21 illustrates the relationship between the projection of ray R 2,3 ^® 
5 projection system and the X-Y calibration surface of the. calibration station. 

Figure 22 is an illustration showing fee relationship between the unknown distance 
Z ' of the object from the scanning device and the locations where ray R is imaged by the 
electromc' imaging device at distances Z{ and Z2. " 

Figure 23 is an illustration showing the coordinate system that used for the X-Y 
10 caUbration surface in generating the entries for the calibration tables for the scanner. 

Figure 24 is an illustration of a first calibration table for the scanner after completion 
of the Z cahbration step; 

Figure 25 is an illustration of a second caUbration table for the scanner after 
completion of the X-Y calibration step. The entries in the second caUbration table of Figure 
15 25 are used to complete the nun entries in the caUbration table of Figure 24. 

Figure-26 -is- an- iUustration- of the- first caUbration table for_the_sc2am^ a^ 

calculations from the table of Figure 25 have been performed for ray R 2,3 and the results 
entered in the first table. It will be understood that a sunilar calculation from caUbration 
table # 2 (Figure 25) is done for each ray at both distances and the entries ui mm for each 
20 ray are entered in calibration table # 1 (Figure 24). 

Figure 27 is an illustration of a two-dimensional bitmap image of a tooth and 
associated anatomical stmctures captured by the electronic imaging device of the scanner of 
Figure 1, 2, 3 and 4, prior to any signal processing in the scanning node or workstation. 

Figure 28 is an illustration of the image of Figure 27 after pattern recognition and 
25 filtering operations have been performed. 

Figure 29 is an illustration of a single "frame'^ of data, that is, a three-dimensional 
point cloud of a scanned object which has been calculated from a single two dimensional 
image by the patteiiirecdg^ decoding, arid 3-D calculations described hereia. 

Figure 30 is an illustration of the points of the cloud of Figure 29 in which three 
30 adjacent points of the cloud are joined together to form triangle surfaces. 

Figure 31 is a view of the three-dimensional surface formed from the triangle 
surfaces shown in Figure 30. 
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Of Barnes, each W consrsting of a fliree-dimensional point cloud of a scanned object. 
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Each frame is typically generated from a different spatial orientation of the scanner relative 
to the object due to movement of the scanner during image capture, hence &.e frames 
overlap to at least some extent The registration process is used to find a best fit between 
the frames relative to each other, and thereby provide a complete three-dimensional virtual 
5 model of the surface of the object from all of the frames. 

Figure 41 illustrates the normal vectors used in the process of Figure 40. 

Figure 42 illustrates the sxmunation of the noraial vectors from frame 1 to reach a 

net normal vector N net- 
Figure 43 illustrates the vectors Vi and Ni from the process of Figure 40; 

10 Figure 44 illustrates the cross product of vectors Vi and Ni; 

Figure 45 illustrates the parameter R, which is used in the frame registration process 
to discard or filter out points which are greater than a distance R from the triangle surfaces. 

Figure 46 is an illustration of the closeness factor or quality index, measured as the 
magnitude of the net normal vector N ^et. as a fimction of iterations of the process of Figure 
15 40, showing how the quality index improves with successive iterations of the registration 

— process^ — — 

Figures 47A and 47B are an illustration of a cumulative registration procedure, 
which is an alternative to the frame to frame registration procediure of Figure 40. 
Figure 4SA-48C are a flow diagram of a cumulative registration process. 
20 Figure 49 is an illustration of a set of frames illustrating a different order for frame 

registration than .frame to frame registration, with the order based on the location on the 
surface of the object for a given firame relative to the location on the surface for other 
frames. 

Figure 50 is a simplified illustration of a set of frames, showing the order in which 
25 the frames were obtained, with the nei^borliness of the frames relative to other frames 
being the basis for the registration order shown in Figure 49. 

Figure 51 is another illustration of a set of frames, with registration of frames 
perfonned in accordance with the method of Figure 49, with the marking in frames 2;-3- 6 
and 7 etc. indicating that that frame has been registered. The marking is just a way of 
30 illustrating that the computer keeps track of which frames have not been registered, as a 
check to insure that no frames are omitted during the registration procedure of Figure 49. 

Figure 52 is an illustration of cimaulative registration based on the first captured 
fi^me (Fl) as being the base line for all successive registrations. 
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selecting an archfonn for the patient and the computer placing the teeth along the arch 
selected by the user. Figure 60 also shows the various parameters by which the orthodontist 
can adjust the shape of the arch, the distance between the teeth, the distance between the 
molars, and other parameters, so as to provide a unique and customized target situation for 
the patient. 

Figure 61 is another screen shot showing the computer model of the patient's teeth 
in a target situation, also showing the numerous parameters available to the orthodontist to 
customize the tooth position, orientation, angulation, torque, and other parameters on a tooth 
by tooth basis for the target archform. 

Figure 62 is another screen shot showing a front view of the target situation and 
additional parameters available to the orthodontist for moving teefli relative to each other in 
planning treatment for the patient- 
Figure 63 is a screen shot of a target situation for the patient showing the virtual 
tooth in a target position, a set of virtual brackets placed on the teeth, and a virtual archwire. 

Figures 64A-64D are four views of a virtual model of a portion of the dentition 
generated-by-the scanning- system of figure 1 ^illustrating, altematiye methods foiL^^ . 
teeth from associated anatomical structure, e.g., other teeth and gingival tissue, as a part of 
the process described in conjimction with Figure 58A-58F. 

Figure 65 is an illustration of an interpolation procedure that can be used in the 
process described in conjunction with Figures 5SA-58F to fill in holes in scan data from a 
template object 

Figure 66 is a screen shot of a workstation user interface display showing a patient 
records screen which the user access various information regarding the patient, includmg 
dental history. X-ray photographs, medical history, photographs of the face, teeth and head 
and the three-dimensional model of the malocclusion. 

Figure 67 is an illustration of a series of icons that appear on a screen display that 
provide some tools for viewing the three-dimensional model of the patient's dentition. 

Figure 68 is an illustration of a set of icons which are part of the screen- displays, 
which act as a navigational tool and allow the user to manipulate the three-dimensional 
models of teeth and brackets on the display. 

Figure 69 is a screen shot from the treatment planning software showing a set of 
individual tooth objects representing the observed stage of a patient suffering firom a 
malocclusion. 
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Figure 81 is another screen show showing the target stage, with the brackets and 
archwire, showing a tooth moved in the buccal and coronal directions by an amount 
iadicated by the orthodontist, and the correction incorporated into the archwire. 

Figure 82 is another screen shot showing a space management feature by which the 
target situation can be adjusted by specifying spaces between teeth or by extraction of teeth. 

Figure 83 is a screen shot illustrating the simulation of an extraction of a tooth 
number 41. 

Figure 84 is a screen shot showing the user modifying the mesial gap between the 

teeth and showing how the user-specified mesial gap is instantaneously represented on the 
screeiL 

Figure 85 is a screen, shot showing the user modifying tooth position in a target stage 
on a tooth by tooth basis using a bonding correction feature. 

Figure 86 is a screen shot showing a wire tab, which allows the user to make 
changes in the shape of ah archwire without changing bracket position. 
5 Figure 87 is a screen shot showmg a wire offset tab, which allows the user to clwnge 

— thte the-bend-sizerloGation,-etc.,-in-the-wrre 

Figure 88 is a schematic representation of an archwhe manufacturing system shown 
in Figure 1. 

Figure 88A is a.block diagram of the system of Figure 88. 
>0 Figure 89 is a peri^ective view of a moveable robot arm used in the manufacturing 

system m Figure 88; in Figure 89 the gripping tool at the distal end of the arm is omitted for 
sake of clarity to show the various oflier aspects of the arm. 

Figure 90 is perspective view of the robot arm of Figure 89, showmg the movement 
of one of the arm joints and the corresponding motion of the arm. 
25 Figure 9 1 is a detailed perspective view of the gripping tool that is mounted to the 

distal end of the moveable robot arm of Figure 89 and Figure 90. 

Figure 92 is a perspective view of the fixed gripping tool of Figure 88 and the 
grippmg tool of Figure 91, in which an orthodontic archwire is gripped by tiie tools. 

Figure 93 is a perspective view of a magazine of Figure 88 that holds a plurality of 

30 straight archwires. 

Figure 94 is a detailed pCTspective view of the moveable gripping tool grasping one 

of the archwires from ttie magazine of Figure 93 . 

Figure 95 is a perspective view of an alternative arrangement of a moveable robot 
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Figure 96A and 96B ar= perspective view, of alternative magazme constioctions .0 
me magazine of Figure 93. 

o merobot ami of Figure 88. 

Figure 98 is a diagram iUustiating the robot software as it relates to the production 
aow in producing orthodontic archwires. 

Figure 99 is a simptified iDustration a se, of teeth sho™g the oHgin of a 
coordu^te system that i. used to calculate bracket location ^ ^ ^ ^ 

d^^stons forapatient bracket location for the .e«h in a t^et situation detennines 
the shape of an orthodontic archwire. 

Figur. 100 is an iUustration showing the vectors drawn fon, the origin of the 
coordmafe system to the center of the brackets. 

Figure 101 is a perspective view of an orthodontic bracket 

Fi^e 102 is an illustration of a vector *awn ftom the origin of ti.e coordinate 
. 'J^emtotoWktt^ormalve^^^^ 
a t^gential vector T extending in the direction of the slot of the bracket 

X, the tangential distance T, and antitangential distance AT,. 

';'S^l'«*°'«i'nutiix to the values for an individual bracket which describe 
me location of tite bracket and its orieutation. which are used to generate the commands for 
the robot to fonn the orthodontic archwire. 

Figure 105 is an iUustration of a set of points PI P2 t>xt u- ^ 

pomis ri, ±'2, P3, . . . FN which represent a 

of boadntg pomts associated with individual brackets for a patient in a target sitiuttion. 
ae location of the points m the three-dimensional coo«iinate system is taown. 

he d- ?^r''"""°°''"°'*''''°'^''^'--''»^P»»dI'4inwhicha 
bend is placed between points P2 and P3. 

'''^l^AisaniUustiationoffonrpoin.sandacurvedea.edbyaBeziersplirxe.a 
" " ^ bend in the wire b«ween pomts P2 and P3 in 

106A ^r/*^ ' fl«w ^art of an algoritiun to calculate the Bc^er spline of Figure 
106A and the loigth of the curve. 

Figure 107 is a graph offeree as a function of deflection for a workpiece such as a 
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Wire. The graph iUustrates that that when a certain amount of force, Fl, is applied to the 
worlcpiece and then released, a deflection D2 results. When the force is released, the 
amount of remaining deflection, Dl, is less than the deflection observed when the force is 
apphed to the wire, D2, since the wire has elastic properties. 
5 Figure 108 is an illustration of the stresses found in a wire when it is bent. 

Figure 109 is an elevational view of the gripping fingers of the fixed gripping tool of 
Figure 92, showing the origin of a coordinate system used by the robot in bending wire. 

Figure 1 10 is a top view of the gripping fingers of Figure 109. 

Figure 1 1 1 is flowchart illustrating a deformation-controlled overbending procedure, 
10 which may be used to compensate for the elastic properties of the wire demonstrated by 
Figure 107. 

Figure 112 is an illustration showing the overbending niethod set forth in Figure 

111. 

Figures 113A-113E are a series of schematic drawings of the fixed and moveable 
15 gripping tools of Figure 92, showing how they moved relative to each other and gnp and 
release the archwire to place the arehwire in position to. form, a bend b.etween.points.P2 and 

P3 of Figure 105. 

Figure 114 is a schematic illustration showing how the movable gripping tool bends 
an archwire while maintaining a constant distance firom the fixed gripping tool. 
20 Figures 1 1 5 A-1 1 5C iUustrate how a bend may be formed in a series of steps - 

Figures 116A-116D are a series of schematic drawings of the fixed and moveable 
gripping tools of Figure 92. showing how they move relative to each other to place the 
archwire in position to form a bend between points P4 and P5 . 

Figure 117 is an illustration of the points defining a portion of an archwire, and 
25 illustrating a technique for placing bends in wires where a substantial distance exists 
between the straight wire segments. 

Figure 118 is an illustration of a portion of an archwire showing a bend formed 
therein to increase tiie forces applied by the wire when the wire has nearly straightened out, 
e.g., near the end of treatment. 
30 Figure 119 shows the wire segment of Figure 118 installed between two teetii. 

Figures 120 shows a wire with a loop in the wire. 



25 



wo 01/80761 

PCT/US()l/n969 

Figure. I21A-121B m^,ra.e onepossiWe .neaod of fencing loop in m= wire of 
Figure 120. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENT 

Part 1. Oven/iew • 

Figure 1 is an illustration of an orthodontic care system 10 incorporating a scanner 
system 12 in accordance with a representative embodiment of the invention. The scanner 
system 12 includes a hand-held scanner 14 that is used by the orthodontist or his assistant to 
acquire three-dimensional information of tiie dentition and associated anatomical stmctures 
of a patient The images are processed in a scanning node or workstation 16 havmg a 
central processing unit, such as a general-purpose computer. The scamiing node 16, either 
alone or in combination with a back-office server 28, generates a three-dimensional 
computer model 18 of the dentition and provides the orthodontist with a base of information 
for diagnosis, planning treatment, and monitoring care for the patient. The model 18 is 
displayed to the user on a monitor 20 connected to the scanning node 16. 

As noted above, the scanner system 12 described in detail herein is optimized for m- 
vivo scanning of teeth, or alternatively, scanning a plaster model of the teeth and/or an 
impression of the teeth. However, it wiU be apparent to persons skilled in the art that the 
scanning system -1-2- can-by-readily optimized- for-a-variety-ofLother_diagnosti.c and/or 
treatment planning and/or monitoring uses in the medical arena. An example is scanning 
the face or head and planning plastic or orthopedic surgery. It can be readUy adapted to 
virtually limitless number of applications in industrial, manufacturing, forensic, 
archeological, scientific, archival or other applications. 

The orthodontic care system consists of a pluraUty of orthodontic clinics 22 which 
are Unked via the Internet or other suitable communications medium 24 (such as the public 
switched telephone network, cable network, etc.) to a precision appUance service center 26. 
Each chnic 22 has a back office server work station 28 having its own user interface, 
including a monitor 30. The back office server 28 executes an orthodontic treatment 
planning software program. The software obtains the three-dimensional digital data of the 
patient's teeth ftom the scanning node 16 and displays the model 18 for the orthodontist. 
The treatment planning software includes features to enable the orthodontist to naanipulate 
the model 18 to plan treatment for the patient. For example, the orthodontist can select an 
archform for the teeth and manipulate individual tooth positions relative to the archform to 
arrive at a desired or target situation for the patient. The software moves the virtual teett in 
accordance Avith the selections of the orthodontist The software also allows the 
orthodontist to selectively place vutual brackets on the tooth models and design a 



27 



wo 01/80761 

PCT/USOl/11969 

~:«!. T ™ When ^. 

infonnatzon regarding the nafiVnt i . . ui^iai 

patient are , . malocctaon, and a desfred tre^ pto. for the 

patient are sent over the communications medt,m, 

5 customized otthodomioarohwire and IT ^ ^ 
theselactaH, « '°^^''°"'=°'^'-P'^»'»f»f*= brackets on (he teeth at 

Tm Zl*r '^'^•^ ^ of appnance systems can benefit 

server 3^ ^"P-^-^ service center 26 inciudes a cenhal 

Zl t - — turing system 34 and a btacket placemen, mann^g 

system 36. These details are described later. 

Figure 2 is a more detailed blocV.diJi«n^r« ^^^t, 
15 m the orth^Hn « block-diagram of the scamimg system 12, suitable for use 

m the orthodontic care system of Figure 1 n,» • 

caonirina ,hr^^- ■ scanning system 12 is a mechanism for 

capturmg.thiee^ensionalia&iniation-of-an«bi5cfW-wK)?l^^^ 

dentition and surrounding anatomical ""f' ^' ""^ P^' sample is the 

soft tissue TX.. ^''™*^'"^°'"™»P«''°'. -8-. bone and/or 

^ ae scamung system ,2 inch«ies a seamier ,4 which is used for image capture. 

and central processmg umt 44 of flB scanning node or workstation 1 6 

ple^^L W ~ ^ '-'-'^ ^ »■> a sHde 50 which is 

"""^ "•"o^-'. *e Hgh, source 53 

■> 7^T 7TrZ ^'"■"-•-^csahighin.ensi.yfash 
'™ 'l Z ''"^ — ^ -"-^'e flash .amp 

oimensional surface of the obiect Pht^Vi^^ ^ ^ -i 

pattern are set for.H • .. 7 °^ ^'^^ of patterns suitable for the 

pattern ^e set forth m the following co-pending patent applications of Rudger Rubbert et 
al:, senal no. 09/254,755 filed March 9 1 ooo- • , . Kubbert et 
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projection pattern is described below. The details on the optics of the projection system 46 
are set forfli in further detail below. 

The scanner 14 further includes an electronic imaging device 56 comprising an array 
of photo-sensitive pixels. A preferred embodiment is an off-the-shelf, color-sensitive, 
5 charged- coupled device (CCD) of a size of 1,028 X 1,028 pixels arranged in an array of 
rows and columns. The Sony ICX205AK CCD chip is a suitable electronic imaging device. 
The electronic imaging device 56 is oriented perpendicular to a second imaging axis 58, 
which is off-set ftom" the projection axis 48. The angle between the projection and 
imaging axes need not be known in a preferred embodiment of the invention. However, if 
10 the 3D calculations are made in accordance with the parameters of Figure 9, then the angle 
and the separation distance between the center of the imaging device 56 and the center of 
the light source 53 need to be known. 

The angle *P will be optimized during design and manufecture of the scanner 
depending on the desired resolution required by the scanner. This, in turn, is dependent on 
- 15 the degree to which the surface under scmtiny has undercuts and shadowing features which 
- - would-result in-the-faUure of-the-imaging device-to-detect the-projection pattem. J'he.greater 
the angle *F, the greater the accuracy of the scanner. However, as angle T increases, the 
presence of undercuts and shadowing features will block the reflected pattern and prevent 
capture of the pattern and subsequent three-dimensional analysis of those portions of the 
20 surface. Angle If is shown somewhat exaggerated in Figure 2, and will generally range 
between 10 and 30 degrees for most applications. 

The electronic imaging device 56 forms an image of the projection pattern after 
reflection of the pattern off of the surface of the object 40. The reflected patterns imaged 
by the imaging device contain three-dimensional information as to the surface of the object, 
25 and this information needs to be extracted from the images. The scanning system therefore 
includes a processing subsystem which is used to extract this information and construct a 
three-dimensional virtual model of the object 40. In the preferred embodiment, this 
processing subsystem consists of a memory 42 storing calibration information for the 
scanner, and at least one processing unit, such as the central processing unit 44 of the 
30 scanning workstation 16. The location of the memory and the processing unit is not 
important They can be incorporated into the scanner 14 per se. Alternatively, all 
processing of the images can take place in the back office server 28 or in another conoputer. 



29 



15 



0 



WO 01/80761 

PCT/USOl/11969 

° • "^^^ '"^ ^---^ » - -educe .he 

amount of mne ,«iu„«i to gene^te fte aree-dtoensional tafonnation 

- ■ ^ U ^. '^'-^ - ^ ^ «^ -"fac... of 

' XILl f T «"^,^='>«^ - a^y-of data Storage ,oca«on. ^ contain 

^on^ for nt»e„,n, portions of .he proieCon patten. U.at are inaaged by a,e eleottonic 
^^dev.^ 5.. ,.^e,„,ho patte. i. projected onto a calibration sttrface I two different 

^ "Tf ^* "° ^ ^ generation 

and use of ti^e cahbranon table is explained in ferflrer detail below. 

-Xk?„ ssanning_system _ requires-at-l™.!t- ™.- 

processin., ,h.^ • ™ prooessmg unit to p^toST ffiii^^ 

~ ' -age. and ..gi^on of ftames to 

umt(CPU)ofthesoamnngworkstationI6 Therprr^ 

after reflection of the patten, off ,b ^'^"'^""^ ™=8e of the pattern 

inte.pola.ion r^vc to the IT^^l T ""'^ 

derives spatial infom^atio, in Z T ""^^^'^^ " 

nr«ie«ertn,« . . °f P°>°« on object thai reflect the 

projected pattern onto the electronic imaging device 

^ho. hundreds or thousands of inaages are generated .of the projecion patten as 
ZZT 7 T " ^'^'^ '^*"'- ^ ^ 2 -"^-es (i.e.. three 

ptrtr rrr *^ --^^^ - -^^^^ 

ZZ ^enTrr"" """^^ ^ -^-i^ensional W or vir^ n.o Jof tl. 
object Wh«.. hundreds or d.„usands of in^ges of the object are obtained different 



30 



wo 01/80761 PCT/USOl/11969 

perspectives, as the scanner is moved relative to the object, the system generates hundreds 
or thousands of these frames. These frames are then registered to each other to thereby 
generate a complete and highly accurate three-dimensional model of the object 40. 

Stray data points are preferably canceled out in generating the calibration table or 
using the calibration table to calculate three-dimensional coordinates. For example, a 
smoothing function such as a spline can be calculated when generating the entries for the 
caUbration table, and the spline used to cancel or ignore data points that deviate 
significantly from the spline. 

Figure 2 also shows a few other features of the presently preferred scanning system 
12. After the CCD imaging device 56 captures a single image, the analog voltage signals 
from the device 56 are amplified in an ampUfier 57 and fed. along a conductor 59 to an 
analog to digital converter 60. The digital signal is converted into a bitmap stream of digital 
image data. The data is formatted by a module 61 into an IEEE 1394 "firewire" format for 
transmission over a second conductor 62 to the main memory 42 of the scanner work station 
16. The scanning system includes an optical scanner holder 64 for the user to place the 
-scanner after die scanning of the- dentition- is-complete— These details are not particul^y 
important and can vary considerably from the illustrated embodiment. As noted earlier, 
preferably fee scanning system is constructed to provide a minimum of equipment and 
clutter at the chair side. Hence, the scanning work station 16 is preferably located some 
distance away from the chair where the patient sits. The cable leading from the scanner 14 
to the base station and/or workstation 16 could be suspended from the ceiling to further 
eliminate chairside clutter. 

The scanning work station 16 also includes the monitor 20 for displaying the 
scanning results as a three-dimensional model 18 of the dentition in real time as the 
scanning is occurring. The user interface also includes a keyboard and mouse for 
manipulating the virtual model of the object, and for entering or changmg parameters for the 
scanning, identifymg sections or segments of scans that have been obtamed, and other 
features. The scanning station may also include a foot switch, not shown;- for sendmg a 
signal to the CPU 44 indicating that scanning is commencing and scanning has been 
completed. The base station may alternatively include a voice recognition module that is 
trained to recognize a small set of voice commands such as START, STOP, AGAIN, 
REPEAT, SEGMENT, ONE, TWO, THREE, FOUR, etc., thereby eliminating the need for 
the foot switch. Scanner start and stop commands from the CPU 44, in the form of control 
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Signals, are sent to the light source thereby controlling the iUmnination of the lamp 52 
during scannmg. 

The Hght source 52 operates at a suitable frequency, preferably at least greater than 
one flash per second, such as six flashes per second, and the frame rate of the CCD imaging 
device 56 zs synchronized with the flash rate. With a fi^e rate of 6 frames per second, and 
a scanmng motion of say 1-2 centimeters-per second, a large of overly between images is 
obtamed. The size of the mirror at the tip 68 of the seamier influences the speed at which 
scammig is possible. The illustrated embodiment-of the mirror at the tip 68 is 18 mm 
square. A larger mirror reflects more surface of the object and enables faster scammig. A 
smaller mirror requires slower scammig. The larger the mirror, the more difficult in-vivo 
scammig becomes, so some trade-offbetween size and utility for m-vivo scanmng exists. 

This overlap between images generated by the scanner 14, and resulting three 
dmiensional frames, allows a smooth and accurate registration of frames relative to each 
other. The frame rate and permissible rate of seamier motion wiU depend on many factors 
and can of course vary within the scope of the invention. Flashing a high intensity flash 
^^P^^-AbnefperioAcf-time is a^^^^^^^ 
exposure time of the CCD imaging device 56 to reduce blurring since relative motion exists 
between the seamier and the object. A high intensity lamp is desirable to achieve sufficient 
signal strength from the imaging device. A preferred embodiment uses 5 Msec flash times 
^0 with smnlar exposmre periods. An alternative embodiment would use a constant 
lUummation source of high intensity, and control e^osure of the imaging device using a 
shutter, either a physical shutter or using elecfronic shutter techniques, such as draining 
charge accmnulating in tie pixels prior to generating an image. Scamiing using longer 
exposures wbuld be possible without image blur, using electronic image motion 
compensatio^ techniques described in Lareau, et al., U.S. Patent 5,155.597. 

Figi^e 3 is a perspective view of a hand-held seamier 14 uled to acquire information 
of an object mider scrutiny, suitable for use in the orthodontic care system of Figure 1. THe 
projection system 46 and the electronic imaging device 56 of Figme 2 are contained in the 
housmg 65 for the seamier. THe housing 65 is sized and shaped to be held in a hmnan hand, 
nie scanner 14 includes an elongate distal portion 66 having a tip 68. The tip 68 is sized 
and shaped such that it can be inserted into and moved within an oral cavity of a hmnan so 
as to enable scamiing of anatomical stmctures inside the oral cavity. A heating mirror (not 
shown) IS placed on the underside of the tip 68 to direct the projection pattern from the 
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optics of the scanner onto the object and to direct the reflected pattern from the object 
towards the imaging optics 108 of Figure 7 associated with the electronic imaging device 
56. The mirror housing has A/C conductive heatmg coil that heats the mirror. The mirror 
is heated to approximately 40 degrees to prevent fogging of the mirror while the scanner is 
5 used in-vivo. 

Figure 4 is an iUustration of a patient 70 being scanned with the hand-held scanner 
14 of Figure 3. The checks and lips are retracted from the teeth and the tip 68 of the scanner 
is moved over all the surfaces of the teeth in a sweeping motion at a velocity of perhaps 1-2 
centimeters per second. The entfre upper or lower jaw may need to be scanned in a senes of 
10 scans, one for the left side, one for the right side, and one for the front. These individual 
scans are registered to each other as described below. Voice commands or activation of the 
foot switch (not shown) indicates when each scanning segment is initiated and terminated. 
The entire process takes just a few minutes. Depending on the color and translucency of the 
object and the illumination intensity and frequency of the Ught source m the scanner, it may 
15 be necessary to apply a very thia coating of a bright reflective substance such as Titamum 

Dioxide to- the object .. 

While Figure 4 illustrates ih-vzvo scanning of a human patient, the scanner can of 
course be used to scan a plaster model of the dentition if that is preferred, or an impression 
taken from the patient. When scanning an impression or a plaster model, the scanning may 
20 be formed in a single pass, without the need for registering scan segments to each other. It 
is also possible that a scan of a patient may be partially taken in vivo and the remainder 
from a model or an impressiorL 

Figure 5 is a block diagram of the back office server of Figure 1 showing the 
elements used to calculate the digital model of the patient's dentition. After the scanning 
25 workstation has processed all the images captured by the scanner and generated a set of 
three dimensional frames, the frame data is transmitted to the back office server 28. The 
back office server 28 performs a cumulative registration process for the frames and ultimaty 
generates and displays the digital model on a screen- display 30; - The raw scanner data m 
the form of three-dimensional frames is stored in the main computer memory 72. The 
30 frame data for N captured images, i = 1 . . . N from the scanner is stored in the hard disk 
74. The hard disk also stores a set of (N -1) transformation matrices [T] j , for i = 2 - 
The transformation matrices basically contain information as to how each frame of three- 
dimensional points needs to be translated and rotated in a three-axis Cartesian coordinate 
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system in order to be registered with the other frames in a best-fit manner. One of the 
frames, such as the fir.t frame in the series, is a starting point for registration and no 
transfonnation matrix is obtained for that frame. The generation of the transfomxation 
matnces, and use of the matrices with the frame data to generate the three dimensional 
model, IS described in further detail below. 

^- Three-Dimansionfll Tmat^ g Genfirarinn 

general- introduction -and" overview in mind,- a presently preferred 
process of capturing two dimensional images with the seamier and gener^on of a tfaree- 
dmiensional model for each image will now be described in detail in this section Figure 6 
IS a flow diagram illustrating the sequence of steps used by the processing unit 44 in the 
scamung station 16 (Figure 2) to calculate three-dimensional information of an object for a 

smglemxage captured by the scamxer 14. This process shown in Figure 6 is performed for 
each image. 

This process can be executed in one processor, or may be perfomied or shared by 
multiple DSP prpcesso^^^ access to- a memory storing tfie-captui^dl^o:drii-ens5^na 

bitmap unages from the scanner, e.g., memory 42 of Figure 2. The type of processor is of 
course not important. The goal of distributed processing is to reduce the time required for 
processmg the scamied image. For example, one processor could perform the cross- 
correlation process described below, another perfomi the pattern recognition, another 
decodmg, and another 3-D calculation.- These processes could be performed independently, 
each process,associated with independent files shared on a disk memory. The processor 
assigned ^,one or more of the processes of Figure 6 accesses the files as needed. For 
"^^IS^ ^^^g P-'^-^ - written on a disk or memory available 

to the other processing units. The processing units are on a network in this example. The 
processor assigned to '3-D calculation access tie decoding output file and writes the result, a 

• set of coordmates for N frarnes and N-1 transformation matrices, onto the disk or memory. 
Any one -of the processors assigned to perform registration could then access the N frame 
data and.the N^l transfomiation matrices and perform the registration procedures described 

below. 

The process of 'Figure 6 consists of four principal steps: a pattern recognition 
process 80, a decoding process 82, a process 84 of derivation or calculation of tfaree- 
dmiensional coordinates for each point in the detected and decoded pattern, and finally a 
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Step 86 of Storing the three dimensional coordinates in a memory, such as the memory 42 of 
the scanning work station 16. Again, this process is perfomied for each captured image 
during the scanning process. In a typical scanning scenario, hundreds or even thousands of 
images may be captured, hence this process of Figure 6 may be perfonned hundreds of 
5 times. The result of the process is the storage of a large number of sets of three-dimensional 
coordinates, each set or "frame" associated with a single captured image. The registration 
of these frames relative to each other to generate a complete virtual model of the object is 
described in Part 3 of-this document. 

As the scanner is moved over the dentition, the imaging device acquires a series of 
10 bitmap images. The acquired bitmaps are analyzed using pattern recognition. Pattern 
recognition detects the median lines of the projected lines, endpoints of flie lines and the 
centers of the colored dots. Other types of patterns are of course possible, such as usmg 
triangles, squares, or other codmg features. The coding is in the vertical direction (in the 
direction of the parallel Hnes), since the distortion of the projection pattern provided by the 
15 surface of the object is in this direction, as explamed more fully m the Rubbert et al. patent 

-appUeation serial no.-0-9/560,131.med.April.28,.2a0.0,.incorppratedbyje — 

The pattern recognition process uses sub-pixel-precision. The color of every dot is 
analyzed as well. Based on the knowledge of the pattern structure and using the colored 
dots, the origin in the pattern for every recognized line is determined. This is necessary, as 
20 significant portions of the projected pattern may not be visible to the imaging optics due to 
shadowing, undercuts and un-sharp areas. A two-dhnensioiial to three-dimensional 
conversion algorithm uses the knowledge of the origin of each imaged line with respect to 
the pattern to compute three-dimensional coordinates of the pomts in the object. As the 
lines are often captured only as fragments, the decoding algorithm does not always have 
25 sufficient information on each line to unequivocally assign that Une to the pattern. The 
algorithm therefore exanune several scenarios of possible affiliations and looks for 
conflicts. In this way the mconsistent scenarios are filtered out. The lines m the projection 
pattem do not change their order in the image. For example, if the lines in the projection 
pattern are sequentiaUy numbered 1-80, and Une 43 is to the left of line 44, in the captured 
30 image hne 43 will be always be to the left of line 44 and never to the right of line 44. 
Inconsistent scenarios are indicated where the order of lines is violated. The correct order of 
the lines can be deduced by a suitable algorithm that examines the scenarios based on line 
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order and etoinate. ali fcose where conflicts or inconsistent line ntunberings exists. A 
unique solution will be foxmd. 

While the prefeixed embodiment of the thiee-dimensionai conversion algorithm is 
based on a sophisticated calibration process and does no, make use of any toowledge of the 
opt, cai parameters, an altemative embodin^ent could nse genial principle of analytical 
.nan^atton assunung that we do have snch knowledge. Analytical triangnlation will be 
explanaed with reference to Figure 9. The projecting device ptojecte a pattetn of 
d.s.mgn,shable elements^ onto a surfece. Ms p^^jected patteni is imaged. For each captmed 
element ts mus, be possible to teU its origin a. thepattetu Ms aUows us to dctennine the 
angle oc between the optical axis of the pK>jeca„n device and .he one ray that has projected 
that element lie location of the pixel at d,e CCD chip that has captured that element aUows 
us to determine the angle p between the optica, axis of the imaging system and the one ray 
tha leads from the projected element at the surfa« to the CCD pixel. Knowing those two 
angles, the angles of tite two optical axes witi. respect to basehne and the length of flte 
baselme, allows us .o calculate ti,e spatial position of ti,e element relatively to the sc^g 
.de»ce./nns calculation-oan-be-done-fcrWderaid^l^^ 
and leads to a plmality of tht^e^ensional points. 1, is important ,„ undet^timd ti.a, as a 
result from this calctilation we receive an undistorted. t^e to scale representation of the 
smftce. WhUe every two dimensional image shows distortions due to parallax effects, the 
« tiiangulation process eliminates this efiect 

The analytical tiiangulation meftod requires precise knowledge of optical 
pamme.^.. Tt. prcf^ embodiment using a calibration table for ti« seamier does not 
require flus knowledge: 

^ gcannerManufactnrP ar,H CalihrarinTi 

Before desoibing tiae details of ti,e ^cess steps shown in Figure 6. an illustrative 

' of the scamaer ^ itself and itsmamrer „f caBbtation willbe described &st. 

Figure 7 is a cross-sectional view of a Tiftr«/,« i. j , , , 

. ""^^ °^ a portion of the hand-held scanner 14 of Figure 

3. showmg fl. optica, elements of the ptojeotion and imaging aspects of the scamter. ll.e 
scanner mcludes it. fiberoptic cable 51 catrying fl^^,, ^om a flash hm.p in the base 
Ration 54 of Figure 2 to a condenser system consisting of a group of lenses 53A. Light 
fiom me hgh. source 53 illuminates a pattern fotmed in a sHde 50. Wide variation is 
possible m the choice of the nattem a .1 

pattern. A presently preferred embodiment is described 
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subsequently in conjunction with Figure .11. The pattern is directed by a projection lens 
system 102 out of the scanner to the minor housed at the tip 68 of the scanner (Figure 3) 
and towards the object under investigation. The scanner further includes a several LED 
light sources 106 (e.g., 2 or 6), which are used to provide general illumination of the object 
during scanning to assist the user in scanning the dentition. A prism 104 surrounds the LED 
light somxjes. The axis of the projection system is shown as axis 48. 

The projection pattern is reflected off of the object, reflected by the mirror in the tip 
68 and received by an imaging lens system 108 centered about an imaging axis 58-. The - 
received pattern is reflected off a minor 110 onto the CCD electronic imaging device 56. 
The CCD 56 produces a voltage signal for each pixel in the device. The level of the signal is 
an indication of the level of Ught impinging on that pixel, aUowing an image to be produced 
from the CCD. The signals are read out of the CCD using known circuitry and amplified. 
The amplified analog signal is collected and transmitted along conductors 59 to Ihe base 
unit for conversion to digital fonn. The signal from the CCD is converted into a colored 
bitmap image in the iUustrated embodiment. Color is used in the projection pattern m the 
illustrated embodiment, fherefore-a-GGD chip is-selected-which.can.detect_cQlors,_ A black 
and white system is of course also possible. 

In the illustrated embodiment, the separation distance between the light source and 
the projection pattern is not known or needed, nor is the angle between the axes 48 and 58. 
However, some non-zero angle between the axes is required in order to obtain depth 
information. The angle selected will depend on the type of surface or object the scanner 
will be used for. These types of implementation details will vary considerably depending 
on the application. Furthennore, it is possible to make the two axes 48 and 58 completely 
independent of each other by locatmg the projection and imaging in separate, independently 
moveable devices. This is described in more detail m the patent application of Riidger 
Rubbert et al, serial no. 09/254,843, die contents of which are incorporated by reference 
herein. The caUbration procedures described herein are of particular advantage when the 
projection device and the imaging device are in two separate, independentiy moveable urtits. 

Figure 8 is a perspective view of a presently prefened scanner calibration station 120 
that is used to caHbrate the scanner 14 at the time of manufacture. The purpose of the 
station 120 is to obtain data for a caUbration relationship for the scanner, such as a 
calibration table, which is stored in tiie memory of tiie scanner system. Some variation in 
the design of the caUbration station is possible; the principle of operation of tiie caUbration 
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station fa that the seamier is calibrated by projecting the pattern onto a reference obj«:t of 
precisely kno™ geometry at tvvo known distances (in the Z direction) and known spatial 
extent (X and Y directions). A planar reference object is preferred, bm it is th=oreticaUy 
possible to use any object of known geometry, but more computationally complex to use 
such objects. The goal of the caBbration procedure is to compile a se, of infonnation (e.g., 
m the form of a table) that completely calibrates the scanner and places it in condition for 
use to scan objects of unknot smface geometry, without the need forpiecise knowledge of 
the-mechaaical-^r -optical properties of the-scamier. While the -present embodiment 
descnbes a caUbration table as Sie result of a.e cahbration process, the infomiation may be 
m odier'equivalent fonns, such as mathematical ^lationshlps or formulae between pixel 
address and dstance, which are operated on to derive distance information at the time of 
use. 

Before ' discussing the presently preferred cahbration device and caUbtation 
relationship, a discussion of the principles of the caUbration invention will be set ferth fcr 

15 ease of understanding. 

. --^^P-^Si'S.device-as disclosed- inTMs-a^ 

mfonnation, but only 2D infenmxior. as the CCD chip is a2D imaging device. Informadon 
on the 3 dimension Aerefore^ in an extra processing step. The 

additional information fliat we can use to perfonn such a 3D calculation is the spatial 
arrangement of flre optical cmnponents, such as shown in Figure 9A. Figure 9A shows a 
schematic arrangement of cpmponents. One portion of the pattern is indicated with a small 
square, and the ray along which this portion would be projected is also displayed. The point, 
where this r^ would tatersect with the surfece of the object, is displayed as a circle. At Uie 
center of this circle, the portion of the pattern would be pi^jected onto the surftce of ib^ 

object. One of the reflected rays wUl mn thronoi, ti,^ • • , 

idys wm run tbrough the imagmg lens system and such be 

projected onto Are sur&e of the CCD chip, which will cause a corresponding signal, to this 
figure, only the center rays are di^layed. e. g. the rays that nm through the center of the 

- lens systems; 

Assunung that there is precise knowledge of the geometrical arrangement of the 
components, it would be possible to precisely calculate the spatial coordinates of the part of 
the suxfece of the object that reflects the considered portion of the pattern. This calculation 
is only possible under three preconditions: 
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(i) The geometric airangement of all components of the scannmg system must be precisely 
known, 

(ii) The exact characteristics of all components must be known (true x/y-coordinates of all 
CCD pixels, including precise knowledge of the dimensions of the pattern; and 

5 (iii) The lens systems must be 'ideal' which means that the center ray must be an ideal 
straight line. 

In mass production scenario for a scanner, it will be almost impossible to guarantee 
these preconditions. One-possible approach would be to calibrate the individual devices, 
- which means that the deviations of the characteristics from the ideal configuration are 
10 determined and noted ("compensative cahbration")- The 3D calculation will then base on 
algorithms like described before, but will additionally take into account known deviations to 
compensate for individual characteristics of each device. However, this compensational 
calculation has to be set up very carefully, and errors in terms of plus/minus signs will not 
easily be detected especially when the deviations are minor. 
15 Another challenge is presented by scanning devices like disclosed in 

PCT/DP^7/01797 by Rubbert, where imaging device and projection devic e a re no t 

physically connected to each other, and therefore the geometrical relationship may be 
completely unknown. 

The calibration procedure that is described herein does not require any pre- 
20 knowledge of any dimensions of the optical and mechanical components, and thus can be 
termed "independent calibration". Furthermore, even any knowledge of the angle formed 
by the two optical axes (angle of triangulation) is not required. 

The background of this procedure can be described best by again just looking at one 
specific portion of the pattern that is being projected along a ray onto the surface to be 
25 measured like indicated m Fig: 9B. It is unportant to understand that this portion of the 
pattern will always be projected along this specific ray R, which is defined by the optical 
characteristics of the lens system and the arrangement of the pattern and the lens system 
with respect to each other. If this portion of the pattern is being reflected-from any- surface 
sitting in the field of view, we can be absolutely sure that this pohit of the surface will be 
30 . located somewhere along this ray R. However, we do not know at which position along this 
ray the point is located. To be able to determine this, the "independent calibration" method 
basically takes samples. The first sample will be a surface that is located at a certain 
distance firom the projection device. The reflected portion of the pattern will show up at a 
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. specific pixel at the CCD chip (or perhaps over several pixels depending on the size of the 
portion of the pattern). It is important to understand that every surface that is ever being hit 
by this ray at this Z-distance will always cause a reflection of this ray R directed to this 
pixel (we are dealing with diffuse reflection, so every point bemg reflected will send rays 
mto many directions, regardless of the angle between ray and surface). This impUes that 
every tune when this portion of the pattern is bemg reflected onto the pixel, we know 
exactly that this point of the surface is located at distance Z,. 

- - Knowledge-of the distance between Z, and the scanner isnotTequired as long as the 
scanner wiU not be used as an absolute measuring system. If we want to use the scanning 
system as an absolute measmiig system, which means that we want to measure the location 
of points relative to the seamier and not only relative to each other, we would then need to 
use the Z-values with respect to the origin of the coordinate system of the scanner. The 
iHustrated embodiment is not an absolute measuring system, but nevertheless generates 
accurate virtual models of the object true to scale. 

During the caUbration process, we wiU acquire a plurality of such "samples" for 
^erent p<,rtiom_oX.^e pattern reflects 

where the relative Z-distances of these levels with respect to each other must be known. It 
will be discussed further below, how many samples will typically be required to receive a 
complete calibration. The result of this sampling proems istkeSxst calibration relationship 
that IS denved for the seamier (1) pixel coordmates for the electronic imaging device for 
nmnerous portions of the pattern, said pixel coordinates associated with distance 
mformation from the projection system in a Z direction at at least two different Z distances. 

Having this first part of the calibration procedure done, we can determine the Z- 
component of every part of the measured surface that is reflectmg the pattern onto the CCD 
chip. However, we do not have knowledge of the X- and Y-coordinates. To get this 
mfoimation. we need to perform the second part of the cahbration. 

Again, we will take "samples", but this time we wiU not make use of the pattern that 
IS bemg projected onto the object during noirnal use of the seamier (the projection unit wiU 
be switched off). Rather, hnages are obtained of a reference object in the field of view that 
IS eqmpped with features of a known geometry, Le., known X-Y spatial relationship. The 
smq,lest implementation would be a point hi Figure 9C, such a feature is a plurality of 
cross-hanrs. The feature being projected onto the CCD chip will show up at a specific pixel 
(or several pixels depending on the size of the feattire). The XA^-coordinates of the pixel at 
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the CCD chip are then assigned to the XA^-value of the location of the feature (and assigned 
to a certain Z-distance). 

It is obvious that if the feature is being moved in Z-direction, the location of the 
projection of the reference object at the CCD chip will change. We therefore have a 
dependence on the Z coordinate, which signifies that the Z-location of the feature must have 
a known reference to the Z-Iocation(s) of the surfaces that have been used in the first part of 
the cahbration. For instance, such a feature could be located at Z,; other features might be 
located-with aimown reference to Zf. ' 

The first feature being captured in this manner would serve as a reference. A certain 
X- and Y-value (in mm or inches) would be assigned to the location of this feature. If such a 
feature would be placed close to the optical axis of the hnaging system, it would be 
preferable to assign X=Omm and Y=Omm to this location. If we want to use the scanning 
system as an absolute measuring system, which means that we want to measure the location 
of points relative to the scanner and not only relative to each other, we would then need to 
use the XAT/Z-values of this feature with respect to the origin of the coordniate system of 

— thescanner; ~ — 

During the caUbration process, we will again acquire a plurahty of such "samples" at 
different X- Y- and Z-locations, where the relative X-, Y- and Z-values of the locations of 
these features with respect to each other and with respect to the Z-values of the first part of 
the calibration miist be known. It will be discussed fiuAer below, how many samples will 
typically be required to receive a complete calibration. 

It is important to understand that the determined relationship between the X- and Y- 
coordmates of any feature being captured and specific pixel coordinates at the CCD chip 
exists only with respect to the Z-coordinate of the feahire. A movement in Z will change the 
XA^-value. Therefore, during normal operation of the scanner, when the caUbration results 
are being used to calculate 3D coordinates, we first have to calculate the Z-coordinate of any 
point on a surface using the caUbration values acquired in part 1, and basing on these results 
we can then perform the XAT calculation, using the calibration results of part 2 of -the 
calibration process. 

There are several options with regard to the number of "samples^' to take during 
caUbration and the way how the results may be stored. The most straightforward approach, 
would be to coUect pixel coordinates for at least two Z-levels of the projected pattern. The 
number of pixel coordinates will depend on the resolution of the pattern. The Z-levels wiU 
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pr.f=rably be defined wifl,in the depfl, of focu. of the sc^er projecioa lens systems and 
unagtng lens system., bu, close to the boundaries of this depth. Having collected pixel 
cootdmates for at leas, two levels, would allow for interpolation of all odaer Z-levels. Part 2 
of the cahbration procedure could also comprise feau^res (points) distributed evenly ac^ss 
. the field of view, and those features could agafa be placed at two different Z-l.vels. which 
would allow for an easy intetpolation of X- and Y-yalues. The pixel coordinates acquire! in 
both parts of the calibntfon process could in the simplest ^mment be stored in a table. 

However, this-straightfcr^ approach has certain disadvantages. First of all, an 
apparatus is required Ofl.erwise i, would not be possible, to place the surfeces required for 
part 1 m a controllable manner with respect o each other, and the features being captured in 
part 2 . also need to be precisely placed with n=spac to each other and to the calibration 
surftce used in part I. Usage of such a cahbration apparatus is no, a problem within 
mdustrial production environment Bu, if scanne,. need U> be caUbra,ed for instance 
orfl^odondc office, it is not recommendable to always ship such a device to the locatio,. 

^"'*'=«'>°<'-=«i to calibrate each portion of fte pattern in various Z-levels. If a 
j= -d, that comprises 3uriaces-at-differen.-Z-levela.-pomo,^-<,Ya-5 pgte^^ 
projected onto levels that are closer to the scanner, and portions wiU be projected onto levels 
that are ftither away. It is well possibly to interpolate also fl,e pixel eootdfaates ftat are no. 
acquired during caUbration. 

Assuming that portions A and C of the pattern wiU be projected onto level Z„ while 

portions C and D will be projected onto levf.1 7 r.r •« 

P ojci-iea onto level Z2, we wiU receive pixel coordinates for 

porhon A and C assigned to Level Z, (x„ and y^ for A, x^, and y„ for C) and pixel 
ooordutates for portion B and D assigned u, Level (x., and y,, for B. x,. and y„ for D). It 
ts wen possible to Unearly int«polate for instance x„ (which has not been acquired) ftom 
and y,. In the same manner y„ could be interpolated fiom y„ and y^. Anoa«=r way to 
recetv. cahbration values that have no. been acquired dixectty would be to draw ti.e 
acqu^ pi..el coordma.es for a certain Z-level onto a sheet of paper aad «,en to con^ a 
best-fit hne (either straight or curved) toougjj .hose points. If themafliematical Amotion of 
tins best-fit line is stored, the pixel coordinates can be calculated using fl«, function instead 
of stonng mem sepanaely. He operation of detennining a best-fl. hne can of course also be 
done directiy in the computer. The best fit line concept is iUustrated in Figure 9C 

Thts procedure would wo* as weU f<. part 2 of the cahbration procedore where 
Ptxel coordipates are being acquired for specific feamres assigned to X-. Y- and Z-values. 
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Again only a subset of features has to be captured at each Z-level, and the remaining values 
can be interpolated in the way described above. It would therefore also be possible to use 
just one calibration device that provides surfaces at least two Z-levels to perform part 1 of 
the caUbration and comprises features at those surfaces that allow for part 2. The density of 
5 portions of the pattern, i.e., features to be captured, depends on the optical quaUty of the 
components of the scanner. We should capture at least four portions of the pattern, 
preferably close to the comers of the CCD imaging device 56 to provide a rehable 
interpolation. 

The advantage of this calibration process is that it requires absolutely no pre- 
10 knowledge of the mechanical and optical characteristics of the scanner and automaticaUy 
compensates for irregularities of the optical components, this including the CCD chip and 
the pattern. It is therefore useful to calibrate scanners that are made from cheap parts, and m 
can be used on scanners that have no known relationship between the imaging and Ihe 
projection device. 

15 

With-the foregoing- discussion- of-the -principles- of ..the. invention . m. mind^ 

representative embodunent of a scanner caUbration device and method will be described 
with particularity with reference to Figure 8. The presently preferred scanner calibration 
system includes mount or holder 122 for holding the scanner fixed in position durmg 
20 calibration. The holder is affixed to the top of a table 124. A caUbration apparatus is 
positioned directly in front of the scanner 14. The caKbration apparatus consists of a Z- 
direction earner 126 havmg one portion 128 fixedly mounted to the table and a second 
portion 130 which can move back and forth in the Z direction between two different 
positions Zl and Z2. An X-direction carrier 131 is mounted to the moveable portion 130 of 
25 the Z-direction carrier. The X-direction carrier consists a first portion 132 which is mounted 
to the moveable portion 130, and a second portion 134 which is moveable in the X direction 
relative to tiie first portion 132, as indicated. 

The X-direction carrier 131 has mounted to its upper surface 136 two calibration 
devices: (1) a smooth, planar caHbration surface 138 used for calibration of the scanner m 
30 the Z-direction, and (2) an X-Y calibration surface 140 used for caUbration of the scanner m 
the X and Y dnection. The X-direction carrier also contains a Ught 142 for providing back 
illumination of the X-Y caUbration surface 140. 
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configurations for a caUbration surface are 4)ossible. In fact, curved surfaces or sloping 
surfaces could even be used, but the simplest surface is a planar surfece oriented directly at 
the electronic imaging device. 

Thus, in one possible alternative embodiment of the invention a calibration device is 
provided for a scanner projecting a pattern onto an object and receiving a reflection of the 
pattern off the object. The calibration devices comprise a calibration surface 144 receivmg 
said projected pattern coinprising two or more parallel surfaces (e.g., 144 and 144") of 
knlSWii separation distanee-aiid-spatiaii extent aaid a pluraUty of point sources of light 143 
provided in the two or more paraUel surfaces. As described herein the point sources of Ught 
are apertures which allow light to pass through the surfaces 144 firom the light source 142, 
but other configurations are possible. For example, the point sources of Ught could be Ught 
emitting diodes arranged m an array in the surface 144. The apertures 143 are formed in a 
precise and known spatial relationship relative to each other, such as by fonning the holes 
with a precision high powered laser on a sheet of metal. Alternatively, instead of apertures 
; 143, black dots could be formed on paper using a highly accurate printmg process, and the 

- -black dots imaged-by the GGD 56 

The calibration procedure described herein represents an alternative, and more 
prefored way of computing three-dimensional information for images as compared to pnor 
art methods. Figure 9 is an illustration of the relevant parameters that can be used to 
D calculate surfece configuration of the object m accordance with a known fashion. The 
method of Figure 9 requires knowledge of the separation distance D or baseline between the 
detector and the Ught source, the angle between the axes 48 and 58, and the angles a and p 
shown in the Figure. The present caUbration method and method of calculation of three- 
dimensional information does not require any of this information. The caUbration 
25 procedure compensates for imperfections in the optics in the optical paths, and therefore 
eUminates the need for high precision optics. Further, there is no need for precise 
knowledge of the placement of the scanner relative to the caUbration planes. There is no 
need to know the angle between the axes 48 and 58, the separation distance between- the 
scanner and the object bemg scanned, or any absolute value of the location of the object in 
30 any global coordmate system. The scanner aUows for truly reference independent 
scanning, yet it gives very precise description of flie three-dimensional surface. 
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The caUbration- will typically be performed once during manufacturing, which 
should be enough to last the life of the scanner. However the scanner can simply and 
quickly re-calibrated if the need arises. 

A representative example of the caHbration of the scanner wiU be better understood 
from Figure 10 and the following discussion. Figure 10 is a illustration of an arbit«iy. 
unique ray R „ which is projected from the projection system of the scanner onto the 
smooth, planar calibration surface 138 of Figure 8 during the first part of the caUbration 
procedure.- The -ray-R is-c^turedijy the electronic imaging device 56, with the 
calibration plane positioned at two different distances from the scanner, Zl and Z2. The 
distance between the two locations is AZ. Distance Zl need not be known, however the 
separation distance AZ is known. The separation distance AZ will vary depending on the 
depth of focus of the imaging optics 108 in the scanner 14. 

Figure 10 illustrates a fundamental principle of the technique that is used for 
calibration of the seamier and generation of three-dimensional information of an object, 
which is considered to be an improvement over the calculations required by the method of 
Figure ?- Figure JQ iUustrates.that-when the plane 13^^^ 

impinges on the imaging device at the location 150. When the caUbration surface 136 is 
moved to position Z2. the ray R ^ impinges on the detector at point 152. The pixel 
coordinates for ray R ^ at both positions is stored in a caUbration table. In actuaUty, the 
pixel coordinates for a large number of rays from the projection pattern are stored for Zl 
and Z2. These pixel coordinates, along with X and Y dimension measurements from a 
second part of the caUbration procedure," give all the information needed to create a 
calibration table necessary to compute three-dimensional coordinates for an object that has 
been scanned with the projection pattern. 

Ray R^„ corresponds to a single point in the projection pattern. Knowledge of 
where in the projection pattern ray R ^ originated from is required. Hence, some pattern 
recognition and decoding of the detected pattern is needed to identify the specific portions 
of the pattern that are being imaged by the various portions of the CCD electronic imaging 
device. To understand the pattern recognition process, the reader is directed to Figures 11, 
12, 17, 18 and 19 and the following discussion. 

Pattern ReRognitinn 
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Figure 1 1 is an illustration of a portion of a pattern that is projected from the scanner 
of Figure 3 onto an object (including the calibration surface 138). The projection pattern 
comprises an array of paraUel lines 156 separated from each other by red, green and yellow 
colored dots 158, it being understood that other types of projection patterns are possible. 
The sequence of the colored dots 158 vary along the lengtii of the hnes 156, and in the 
direction perpendicular to the lines. This technique is used such that using pattern 
recognition and decoding processes, described herein, every region in the projection pattern 
caH-be -decoded fi-om-the-ptsel-data- from the-imaging device. Figure-l-l-illusti:ates tiiat Ray- 
^„ can be taken to have originated at the intersection on one particular line and one 
particular colored dot, and this location can be determined precisely form the pattern 
recognition and decoding process. Similarly, the ray along the line L one row below ray R 
^ can also be identified. In the present example, the pattern is constinicted such that there 
are N colunms or hnes, and M rows of colored dots. Figure 12 illustrates showing that the 
various rays of Ught passing through the pattern of Figure 11 can be represented by an array 

15 ofNXMpoints. 

- "Iliis-array of points representing the-projection-patternof Figure J lisjina 

electronic imaging device or CCD 56 arranged in an array of row and colunms, as shown m 
Figure 14. There are X columns of pixels in the X direction and Y rows of pixels in the Y 
dkection. In the illustrated embodiment there are 1,028 pixels in each direction. 
20 Figures 17, 18 and 19 illustrate the pattern recognition process for captured two- 

dunensional images. Figure 17 shows the signal level from one row of pixels in the 
electronic imaging device. The signal indicates that line L of the projection pattern is 
imaged at pixels 20-23 of a particular row of the pixels. Averaging of the pixel values over 
the four pixels allows the center pomt of the hne relative to pixel 21 to be calculated, vnth 
25 sub-pixel resolution. Figure 18 shows how the Une L and adjacent colored dots are imaged 
on the surface of tiie unaging device in this example. Note that the line is not necessarily 
centered over any column of pixels and hence the averaging must be performed to determme 
the colter point in the Une. A similar pattern recognition process is performed for the 
colored dots, as indicated m Figure 19. The center of each colored dot is located, as is the 
30 center points of the lines for every line and colored dot unaged by the imaging device. 

The pattern recognition process thus takes the output signals of the unaging device 
(m the form of a colored bitinap image) and retams a set of pixel locations for centers of 
hnes and centers of particular colored dots. The ne.xt step in tiie process is correlating these 
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pixel locations for lines and colored dots to particular lines and colored dots in the 
projection pattern. This process is referred to as decoding (process 82 in Figure 6), and is 
described in detail below. Decoding is not nonnally needed during the calibration 
procedure described in conjimction with Figure 8, since thcZ caUbration surface is planar 
and the arrangement of flie projection pattern on the CCD 56 is preserved. Decoding is used 
however during use of the scanner to scan an object of unkno^vn surface configuration. 

The decoding process is the process of converting a set of pixel addresses for lines 
imaged by the imaging device, and a set of pixel addresses for particular colored dots 
imaged by the imaging device, to particular lines and colored dots in the projection pattern. 
Decoding is not absolutely required during cahT^ration (particularly where the Z caKbration 
• surface is a planar surface). It is used, however, during processing of images on an object 
having undercuts, shadow features, or other irregularities. It may be possible to decode 
only a portion of the received pattern, since ordering of lines in the projection pattern is 
.-.preserved--For example,-if-lmero-Sid-re^ ^ alsVdecodeT 

smce their spatial,relationship relative to lines 13 and 16 are preserved. 

The imaging analysis process needs to know that a particular pixel is imaging a 
particular line or a particular colored dot. The projection pattern or screen 50 (Figure 2) 
varies continuously in both directions, due to the unique and continually varying sequence 
of the colored dots. The decoding process simply examines wher« the red. yellow and green 
dots are being imaged in the imaging device, and compares these results with the known 
sequence of red, yellow and green dots in the projection pattern, and thereby locates or 
Identifies each ray with ref«ence to the projection pattern. For example, the process knows 
that, for example, pixel 21 in row N of the imaging device is imaging the center of line 13, 
row 55, in the projection pattern. 

"R^erimg again to the caUbration set-up of Figures 8 and 10, the scanner takes two 
images of the Z-caUbration surface 138, one at distance Zl and the other at distance Z2. 
The pixel addresses where each ray R in the projection pattern is imaged by the array is 
stored in a calibration table referred to herein as caKbration table # 1, shown in Figure 24. 
At this point, we know how the imaging of the projection pattern varies as the caUbration 
surface is moved in theZ direction relative to some imaginary plane Zl in front of the 
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scanner. However, the X and Y relationship is not yet known. Therefore, the scanner must 
be calibrated in the X and Y direction using a pattern of known geometry. This is explamed 
in conjunction with Figures 8 and 13-16. 

Figure 13 is a illustration of the X-Y calibration surface 140 of Figure 8, showing 
the array of Q X P points (tiny apertures 143) in the calibration surface 140 being organized 
into a coordinate system having an origin at the center of the surface 140, in the shape of a 
cross 145. The caUbration surface 140 is conceptualized as consisting of four quadrants I - 
-IVT-iFigure23-^hows-oneT)ossible-Tiumberiiig-convention-of-the-points-i^^ 
origin. In the illustrated embodiment, the points of the X-Y cahbration surface 140 are 
actually tiny apertures spaced ftom each other a known distance (e.g., 1 mm). The apertures 
act as a pluraUty of point sources of Ught when the light source 142 positioned behind the 
surface 140 is activated. These points of light are unaged by the electronic imaging device 
56 during the second part of the cahbration step. By counting pixel signals (indicating the 
imaging of a point source in the surface 140) over from the origin in the X and Y directions, 
15 it is possible to detemiine which point in the surface 140 is being hnaged by which pixel, 

again-with- subpixel . resolution. _Since _we.know Ihe address _of_fee pixels which ilh^^Mte _ 

the specific portions of the projection pattern, and we can know the distance from the ongm 
of the surface 140 that this pixel is imaging, it is therefore possible to calibrate the pixels m 
the X and Y directions. A second cahbration table, shown in Figure 25, is used as an 
20 interim step to generate the distance values in the X and Y directions for the principal 
calibration table # 1 in Figure 24. 

This process will be explained by example. Figure 15 is an illustration showmg the 
interpolation of pixel addresses in X and Y directions for a given ray R from a scanned 
object from two points of the X -Y cahbration plane previously imaged by the electronic 
25 imaging device during cahbration. Figure 15 mdicates that a given point in the projection 
pattern, ray R,3, is imaged by some pLxel that is located between the pixels that imaged 
points in surface 140 that are in quadrant HI, between 14 and 15 points to the left of the 
origin and between 14 and 15 pomts below the origin. - This is suggested by- Figure 21, 
which shows where ray R ^ is imaged on the CCD chip. Figure 21 also indicates where the 
30 corresponding points on the X-Y cahbration surface 140 are imaged by the pixels m the 
electronic imaging device. As is shown in Figure 21, in the present example the pixel 
values are between 30 and 90 in tiie X and Y directions. 
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Figure 25 shows the X - Y calibration table # 2 that is used for generating distance 
entries in X and Y directions in the calibration table No. 1 shown in Figure 24. Figure 25 
illustrates that for each point m the X-Y caUbration surface 140. coirespondinj pixel 
addresses for pixels imaging those points are identified and stored in the table. Ihisisdone 

5 for aU points in the four quadrants of the X-Y calibration surface 140. These values are 
obtained when the X-Y calibration surface is positioned at both distances Zl and Z2, and an' 
image is generated at both positions. The entries in the table are pixel addresses in X and Y 
(hrections, expres^ed-in-sub-pixel-resolutionr-Rg,resentative-entries for Quadrant -I- are- 
shown. It bemg understood that entries are made for aU the points in the X-Y cahT,ration 

3 surface. 

Now, if we know that ray R of the projection pattern from the Z-calibration 
procedure (using the caUbration surface 138) is being imaged at a particular location, we can 
use the calibration table # 2 of Figure 25 to compute an entry m mm for table # 1. Again, 
usmg the present example, assume ray R (corresponding to line 2, row 3 in the projection 
pattern) is-imaged by pixel havmg an address of 30.2 in the X direction and 36.2 in the Y 
_ chrechon, at ZJ =_Z1. I^i^- distance m mm can be calculated-from-^-®^^^^^ 

entnes m caUbration table 2 of Figure.25. This is indicated for the entries in Une 2, row 3. 
At the Z - Zl distance, this point is imaged at pixels 30.2 in the X direction and 36.2 in the 
Y dn-ection, which corresponds to a distance of-14.6 mm in the X direction from the origin 
and -14.4 mm in the Y direction. Similarly, at Z = Z2, this point is imaged by pixels 43.0 
m the X direction and 31 in the Y direction, which corresponds to a distance of -14.8 mm in 
the X dnrection and -15.8 mm in the Y direction. This information is entered into table # 1, 
as shown in Figure 26. 

This interpolation takes advantage of a Unear relationship that exists between pixel 
address and distance for objects at an unknown distance Z • from the seamier. This can be 
best appreciated from Figure 22. Since we know that ray R _ is imaged at one point 160 
m the miaging device at Z = Zl, and that it is imaged at another point 162 at distance Z = 
Z2,-theraymust fall-along the dotted line 164 where Z' is between Zl andZ2. Similarly, if 
Z ' > Z2 It lies along the line mdicated at 166. If Z ' < Zl, it Ues along line 168. This 
hnear relationship between distance and pixel address is the key to obtaining Z information 
as to ray R ^ and X and Y distance in mm. Furthermore, since AZ is known exactly 
dunng the caUbration (e.g., 7 mm in the illustrated embodiment), and a linear relationship 
exists between pixel address and distance, the location of exactly where ray R Has along 
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the line 164, 166 or 168 teUs us with precision the position of the unknown object in the Z 
direction, relative to virtual plane Zl. This location of where ray R ^„ is being imaged by 
the electronic imaging device is airived at by the pattem recognition and decoding 
processes described herein. 

5 Refeiiing again to the example of Figure 16 and 17, we know ray R „^ is imaged at 

some region of the CCD, and this is stored in table 1 (Figure 25). From calibration table 2 
(figure 26) we know the X and Y coordinates of neighboring points in the X-Y calibration 
gnd m the region of ray K^By-mmitpms^mnsf-fti^X--^d-Y-^^^^^ 
calculate the distance in mm from the origin since the points in the X-Y caUbration grid are 

10 separated from each other a known distance. This is done for all N X M portions of the 
projection pattern. 

For example, calibration table 1 of Figure 24 tells us that ray R is imaged at pixel 
addresses X = 30.3 and Y = 36.2 for Z = Zl, and at pixel address X = 43andY = 31atZ = 
Z2. We then look to the table 2 entries (Figure 25) to find the closest X and Y points in the 
15 X-Y caUbration grid by looking at the pLxel addresses in table 2. This is shown in Figure 
1-6. T^-Meipolationofthe pixel-addresses in table 2 to-the known pM 
1 results in an X, Y address in mm. In the present example, X in mm = -14.6, Y in mm = - 
14.4. The results are now added to table 1. see Figure 26. The same is done for the distance 
Z = Z2- The same is of course perfoimed for all the N X M rays in the projection pattem, 
20 resulting in a completely fiUed caHbration table 1. Table # 1 in Figure 26 only shows the 
entries for ray R 2. 3, but the process is done for all rays for Z = Zl and Z = Z2. 

Figure 20 Ulustrates the points of the'X-Y caUbration plane from the calibration 
station of Figure 7 at two positions Zl ands Z2, relative to the optical elements of tiie 
electronic imaging device. It will be apparent that when the Z = Zl, some points in the X- 
Y caUbration plane will not be imaged, but will be when Z = Z2. These pomts, indicated by 
AQ in file Q direction, exist also in the P direction. The calibration table 2 takes these mto 
account. Some points in all four quadrants may not be imaged at Z = Zl, but will be imaged 
at Z = Z2 during X-Y caUbration. Points indicated at 170 are imaged at both values-of Z- 

From the above, in one aspect of the present invention, a machine-readable memory 
30 is provided for a scanner used to calculate three dimensional information of an object 
scanned by the scanner. The memory may be in the scanning unit itself in a separate work 
station for the scanner, or in any computing device such as a remote computer that processes 
acquired image data and generates a three-dimensional model of the object. The memory 
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Chrises a. of storage ,ocat,ons con.aMng a caUtaation relation^p for fte 

such as a .ab,e. oa,*..o. .,a«..hip iden^fi. pi.., eoonta.es for 

—us porfons of a pattern projected onto a caHbration surface located a. two different 
d^^ces fou, tt.e scarce, and distance iu^^nnation in X and Y directions for .he persons 
^ me patte™ for me two diffet^t distances. Tie cah^ration entties stored in n.e»ory 

allow the scanning system to comnute three i 
„ ^ ^ ■ ^ '°™=^^™SK>naacoordmatesfer points on an object 

reflectmg the projection pattern onto the electronic imaging device 

.*>w-tha«he.cannertas^eenxompietel„,ted^.„^y.^^^^^ 
^mc^own ^stance and having some unsown surf^e configuration. The derivahon of 
X. Y and Z coordmates for the surfice will be explained in U.e next section. 

■ P'^'^tion ""-P Foint rio„d ima,e t.,^ ^ T'riir ^} 

coordinaTr- "'^^ ^ ^'^ of 

c^^v r ^ ^"-^'^ ~ Table 1 

■ IT • " ' - - ^ve X. Y and Z 

.■l"??*??.'??. for anxjobject^at. an-unknown^distance- Tisr; i. - 

wi,^ ,v, i ■ . distance. The scanner has the most accuracy 

when the distance is between the values 71 7n ^ ... , 

fee, h„, , . and Z2, such that the captured unages are in 

fecus. but ^stances outs.de of this range may stin be able to be iniaged and decoded. 

First aie e ect^nic imaging device 56 captu.es an hnage and the image is subject to 
^pattern recogn.t.on and decoding, steps 80 and 82 in Figure 6. described ° detail Lve. 

coteed dots Of the p.«teni Uia. are projected onto the object and imaged by the pixels of the 
^^777 ^^^"^ -'"^ -i"^ in Table 1 

point in the captured imaged, in three dimensions. 
The process is as follows: 

capturel"' ""'^ ^ "'"^ "^""^ '""J-"- P""^ ^ » 

r rdT "^"^ "^^^ °f 

^= distance. Z ■. for any poin, 

measured flom the virtual plane Zl is as follows >- P • 

Z ' (inmm) = 
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AZx mP:^^i,reH pixe l # - Pixel # for line and row of pattern at Zl 

. . • pixel # for line and row of pattern at Z2 - pixel # for line and row of pattern 

at Zl . . 

where A Z. is. the distance from Zl to Z2 in the caUbration set up described above. 
.... Using ray R, 3 as an example, if this ray is imaged at pixel # 35 in the X direction, 
from table 1 the calculation is as follows 

Z • (in mm) = 7.0 mm x ^5 - 30.2 where A Z is 7 mm. 

43-30.2 

= AZx a Here, a = 0.375, a linear scaling factor. 

Therefore Z ' = 0.375 X 7 mm or 2.625 mm. The point on the object reflecting 
ray R 2^ is 2.625 mm from the virtual plane Zl. The Z value for aU other points m the 
object are also measured relative to a virtual plane Zl . 

Now, Table 1 (Figure 26) is referred to determine the mm values in the X and Y 
dkection for this point in space and anothe r mt erpolation is p erformed to the mm entaes m 
table 1. The calculation is as follows: 

Since we know we have hne 2, row 3 m the pattern (from the pattern recognition and 
decoding process), we need to interpolate the mm entries in table 1. 

X value is between- 14.6 and - 14.8 AX = 0. 2 mm 
Y value is between - 14.4 and -15. 8 AY =1.4 mm 

The true value of X = X - (a x AX), similarly the true value of Y = Y ,tzi " 

x AY). 

Therefore: 

The true value of X for ray R 2^ = - 14.6 - (0. 375 X 0.2) = -14.675 mm 
The true value of Y for ray R „ = - 14.4 - (0.375 X 1.4) = - 14.925 mm 
Summarizmg, the X, Y and Z coordinates for the point in the object reflecting ray R 23 

X=- 14.675 nam . _. . . . ., . _ 

Y = - 14.925 mm 
Z =-2.625 mm 

These points are stored in memory 42 of the scanning work station 16, step 86 of 
Figure 6. This process is performed for every ray R „.„. in the projection pattern that is 
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recorded in the captured image. The result is a set of three-dimensional coordinates in 
space, referred to herein as a "fiame", for each captured image. This set of points can be 
calculated in a small fraction of a second with a general-purpose computer. 

The pattern recognition, decoding and 3-C coordinate, calculation process wiU now 
be explained with reference to tvvo-dimensional bitmap images of teeth. The process 
described below is the same for any object bemg scanned. 

Tart 3: GeiiCTatrori oTDigifal TTT^p r^^•7^»- 

A complete ftree-dimensional model of the patient's dentition can be generated from 
the scanning system of the present invention. The process requires an operator moving the 
scanner 14 '(Figure 3) over the dentition, preferably in-vh'o, and generating a se: .3 of 
frames of images. The frames are obtained at a rate of at least one frame per second as the 
scanner is moved over the teeth. Tbe scanning of an entire jaw may require three separate 
scanning operations or "segments" due to maneuverabihty constraints and a break in.the 
capturing of hnages. While the scanning is occurring, the four steps of Figure 6 are 
¥^ormcd for tiie stream -Ofxaptured-images^-The-end resultis the stofage-df a se^^^^ 
m the main memory of the scannmg work station 16. The frames can be registered to each 
other usmg one or more of a using a variety of frame registration techniques described in 
detail below. Once all the frames have been registered to each other, a complete three- 
dhnensional vfrtual model of the patient's dentition is displayed for the orthodontist. This 
computer model provides a base of information for diagnosis and planning treatment. An 
introduction to the treatment planning aspects of the overaU orthodontic care system is set 
forth in Part 4. 

Figure 27 is an illustration of a two-dimensional bitmap image of a tooth and 
associated anatomical structures caphn-ed by the electronic imaging device 56 of the scanner 
of Figures 1, 2, 3 and 4, prior to any signal processmg in the scanning work station 16. An 
inspection of Figure 27 indicates that the image includes various lines and colored dots of 
the projection pattern, as it is reflected off of the tooth and associated anatomical' structures. 
The location of where these hnes and colored dots are imaged on the imaging device 56 
contains information as to the three-dimensional shape of the tooth and associated 
anatomical structure. 

Refening back to Figure 6, the first step is a pattern recognition process on the 
c^tured image. ■ Figure 28 is an iUustration of the image of Figure 27 after pattern 
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recognition and filtering operations have been perfonned. The filtering operation basically 
returns a zero pixel value for those pixels where the pattern recognition process caimot 
detect the lines and the colored dots, for example due to the surface being out of focus (i.e., 
out of range of the scanner optics). 
5 After the decoding operation is done on the image of Figure 28, the result is a set of 

pixel locations where the decoded rays R ^„ of the projection pattern were imaged in the 
imaging device 56. Step 84 is perfonned for ail such rays, using caUbration table^# 
Fig5rel 26 stored m memory tor th -eTcanner. The resultrst^-6-o^f7iggei67isarset-of three 
dimensional coordinates for all the points in the image, a point cloud comprising a "frame." 
10 Figure li is?an iUustration of a single "frame" of data, that is, a three-dimensional point 
cloud of .a scafined object which has been calculated from a single two dimensional image 
by the pattern recognition, decoding, and 3-D calculations described herein. 

Figure: 30 is an illustration of the points of the cloud of Figure 29, in which three 
adjacent poiiits of tiie cloud are joined together to form tiiangle surfaces. The usage of the 
15' triangle surfaces in a registi-atidnprqcess is described below. Figure 31 is another view of 

the three-dimensional iirfaee formed-from the-triangle surfaces shown in Figv^e^ 

32- is a view of the siiface of Figure 31, smoothed by a smoothing algorithm to give a 
smoother representatipii of the surface of the object. Commercially available off-the-shelf 
" software exists for tiing a set of three dimensional coordinates and displaying them on a 
20 computer monitor, and such software is used to display the three dimensional surfaces (if 
desiredby the user). ^' ^ 

Figure 33 'is another example of a bitmap image obtained by the electronic imagmg 
device of tiie scanner^ Figure 34 is apian view of the three-dimensional surface obtamed 
from the two-dimensional bitinap image of Figure 33, after the steps of Figure 6 have been 

25 performed. . The 

Figure 35 is a perspective view of the three-dimensional surface shown in Figure 34. 
software programs enable the surface to be rotated in any degree of freedom, allov^rmg for 

complete inspection and visual analysis of the surface. ~ 

Smce the scanner and scanned object move relative to each other during capture o 
30 the scanned images, the three dunensional coordinates for a large number of firames will not 
agree with each other. In other words, the X, Y and Z coordinates for a given point on the 
object will change from fi^e to frame since the point was imaged from a different spatial 
orientation for each image. Hence, the frames have to be registered to each other to 
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gen««e a complete ov^-aU digiw „,odel of *e object The present invention provides for 
vanou. registration procednrcs to be perfomed on ttre ftames. ,o find a best -fit solution for 
coordtnates of fte object in one frame vis-a-vis coordinates of the object in oU^er frames. 
These regtstradon proced,n:es are described in flne following sectioa 

Figure 36 is a flow chart iUustrating the steps perfomed to generate a complete 
tee-dunensional model of the dendtion of a padent from a sedes of soana of fte upper and 
^er^aws. The steps include an inidal step 190 of detennining an entry point into a 
regr^don process, a Smrw-SmormmmmT/m^9Z-o-mm^f^,-^^ 
another, a segment registration process 194 te register segments (i.e.. portions of a scan 
Where .he scan was intem^ted for some t^on) to one another, and finally a cumulative 
regrstration procedure 196 to which the flames are registered to all other frames te obtain a 
shghUy mo« accurate model of the object than that obtained by frame to frame regis^tion. 
K wtll be understood drat depending on the application, tire step 194 may not be retpured, a 
fiame to frame registration may only be needed, or tire user may only desire a cmnulative 
regtsti^on and steps 192 or 1 94 are not performed at afl. 
. . '=Si^'ration-is.a-a«e^ensior,al-m^^^^ 

the frames captirred by dre scamrer. An example of such a model is shown in Figure 37A, 
Where surfece 199 mdica.es dre sum te« of points of all dre frames in drree-dimensioma 
space. Frgure 37B iUustrates one small section of dte surface, showing for example die 
pomts m fluee-dimensional space from diiee frames. 

A pr^fared registiation .echmqne involves registering a set of points (diee- 
Amet^onal coordinates) oompdsing a flame to a surface fonned by a p^vious frame (or 
gro,^ of flames), radrer d«m regisfration of one set of points to anodrer set of poin«. Ms 
« due to a relatively coarse projection pattern used in dre iUustrated embodimen.; ttre poir..s 
^ be low m density as compared to dte curvature of dre object Figure 37C shows one 
flame fiom a scan of a toodr, witt, tt.e points in dae flame comiected by lines to fonn a set of 
tr-angle surfeces. The coarseness of die projection pattern (and widely spaced points in the 
pomt cloud) is compensated by die diat a given portion of d>e surfece is c^tirred by 
overlappmg flames, and dre registtation occurs flom die points to one frame to &e surface 
m the previous frame or a surfece defined by more d,an one previously registered frames. 
Tie regrstration process described hereto ultimately pemuts a flne resolution in tire three- 
dnneusional model. Tlis is mdioated by Figure 37D, showmg all dre frames for tire tooth 
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scan registered to each other to create a very fine and high resolution-virtual model of the 
tooth. 

A. Entrv point into Registrati on (stev 1 90, Figure 36) 
5 Registration processes require a starting point for fitting one frame, frame i to 

another firame, frame i + 1. The starting point, in the illustrated embodiment, is rough 
calculation of the offset between overlapping points in the frames in X, Y and Z directions^ 
- mCTeaspri6f"a rt systems i Save good pre-kndwledge oftSel^atiarM-alao^ 
known physical arrangement of the scanner and the object, the present system does not. 
10 The starting point is the initial assumption of spatial relationship between one firame and the 
previous frame (and one frame and a set of previous fiiames). 

The method of calculation of the offset in X and Y directions is iUustrated in Figure 
38A-C. Figures 38A-38C are an illustration of a two-dimensional cross-correlation 
procedure in the X and Y directions. The procedure, along with the procedure in Figures 
15 39 A and 39B, is used to find an initial entry point into a registration algorithm between 

— successive-fi:ames of data; - — 

It can be seen from Figures 38A and 38B that frame i-t- 1 is moved in the X and Y 
directions from frame i. To obtain the amount of movement, AX and AY, the images are 
projected to both X and Y axes by adding all pixel values of every row, acquiring a one 
20 dimensional Y-vector, .and adding all the pixel values of every column, acquiring a one- 
dimensional X-vgctpr. This is performed for both frames, resulting in X a^me i a^^^ ^ ' * 
and Y i and Y i ^ 1 The vectors are smoothed to suppress the influence of the projected 

pattern. 

To compute AX, the absolute value of the difference between each value of the X- 
25 vector of frame i (frame i - X 0 and the X vector of firame i + 1 (frame i + 1 - X ^ i ^ i ) 
is calculated with a varying position shift within a range of -xa < k < +xe. The sum o 
these values represents the resemblance of Xft,„„5and X^^,, shifted by a certain amount 
k. The minimum value of k is determined. This result gives the shift onnovement m. the 
AX direction. 

30 The same process is also performed the Y direction. As can be seen in Figure 3 8C, 

if frame i is moved by an amount AX and AY, the overlapping points in both firames will 
have the same values and the sum of the difference in pixel value wiU be approxmiately 
zero. 

57 



15 



>0 



WO 01/80761 

PCT/US01/J1%9 

the Z onprocedwein 

I-e 200 represent, a three-dimensional surface of the second W. fonned by connecting 
tee adjacent pon^ts in the point Coud. Hie set of points 202 represents the points of the 
«ou representing W I. To contp^e .he Z offie, A Z. the sun. is taicen of aU the 

Z IT f . '""'^ "'^^^ - an average 

-4™"^'- ™«-"-*»=ftrFrantel. The difference between the average Z 
value ofTrame 2 and the aveijve Z val.v. i; ,- 

the ate h«3S,a,io„ of Fta^e , hy a.e anaount AX. AY and AZ. ^he result 
. ^thepotn^ of 1 are ,uite close to the Mangle surfaces of Fraote 2. Ita values 

^""^ ' and used as an entry point to a registration 

B. PrametoFraiT>pPggif;tTatir.n 

Frame to fame tegistradon is a process fer registering one tame with anoflrer 
.fr?-a,fl« r,..fi„du.g a best.fi.-in..ern:.of tr^WSi-aid 

m ^e fanes agree with each o««r. If fl,e fiames are generated in sequential order, ftame 

^ the second iiante. *om a.e four* fane to the third fa.e. etc. Franae to frame 
^hon ^ be performed very ,uicMy. „ can be performed in a manner such that the 
™ of Jhe scannmg system sees the results of fane to fame registration on the 
momtorof the scamnng work station while aey are stiu scanning the patient What .hey 
^1 •^'^^ -P~on of the dentition on the monitc. for those 

^mons Of the dentttion that have been scanned thus fc. As adm«o„nl fanes a« ob.ah.ed. 

TLZuT' '° "^"^ and added to the computer mode,, vn^ sc^ 
2 ompl«e. the computer model can be mtated around on the monitor and ir^spectad to see 
*at all relevant portions of the tee* 5nd anatomical structures have been scamred. The user 

thus gets-mun-ediate feedback on the results of • 

^^^^^ °^ scanning using frame to frame 

registration. 

Figures 40A-40D is a flow charf of a fr,^ * ^ 
. ^ ^ cnart of a frame to frame regisfration process for a set of 

^es, each frame consisting of a three-dimensional point cloud of a scanned object. Each 

I T ™' ' ''''"^^^ ^^^"^ °^ ™ -lative to the 

Object due to movement of the scanner during image capt^e, hence the frames overlap to at 
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least some extent The registratioii process is used to find a best fit between the frames 
relative to.- each other, and thereby provide a complete three-dimensional virtual model of 
the surface of the object from all of the frames. The end result of the frame to frame 
registration is a substantially exact three dimensional model of the scanned object. This 
5 object is represented by a large set of point coordinates in computer memory. The result is 
also represented as a set of transformation matrices providing information as to how each 
fi^e of points should be translated and rotated in three dhnensions in order to fit to the 
previous rrame. 

The frame to frame registration process is also an iterative process. At the end of 
10 each iteration, a comparison is made as to how "close" the two frames are to each other m 
three dhnensions. If they are not close enough (with "closeness" determined in absolute 
terms by a quality index, say m microns), anoflier iteration is done, using the results of the 
first iteration. The frame to frame process may contmue for tens or even hundreds of 
iterations, depending on how fine or precise the user wants the registration to be. The 
15 process stops when a best fit between two frames has been reached or a maximum number 

- — of iterations has occurred. 

Referring now to Figure 40A-40D in conjunction with Figures 39 and 41-43, an 
mitial step 209 is performed as an entry point into the registration procedure. In this step, 
the processes of Figure 39A and described previously is performed to make a Z-coordinate 
20 ti^nsformation of frame i-1 and frame i. Basically, this transformation is performed by 
summing the Z coordinate values of both flames mdividually, finding a median Z value for 
eachframe; finding the difiference or A Z value from these median values, and shifting one 
frame (fi^e i) by this amount to bring the Z coordinates of all tiie points in that firame 
closer to the points of frame i-1. The rationale of step 209 is as follows. The Z axis 
25 measurements of a surface represent a degree of freedom of the scanner movement. 
Scanner movements in the Z direction do not provide significant new information on the 
object being scanned. However, because of the focus optics of the electronic imaging 
device.(i08 in Figure T), the visible surface of the object becomes sHghtly smaller or larger 
depending, on how much the scanner is moved in the Z direction relative to the surface, 
30 wMle the center of the surface (or, loosely speaking, the "center of gravity") of the visible 
area basically remains nearly the same with Z direction movement. The Z coordinate 
transformatioii' step 209 ehminates this effect by normalizing the Z surface coordinates. 
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™. process also makes possible ttie exotoio. criteria of step 2 described below, by wbich 
non-overiapptog points and stray data points ^ excluded fem the registration process. 

The registration procedure itself starts with step 1, 209 in Figure 40A. At this step, 
—n distance vectors 248 (N1,N2.,..) are calculated flon, ev«y point in Franre i to 
the surface of fianre i-1. The surface of a flatne can be obtained easUy by connecting 
nerghborhood points to together with a triangle or other polygon. The nunimum distance 
jector^r eachpont, in Frame i, is defined as fte distance vector having a magdtude which 
rs the nnnin.un> ofim^mo^^Ssim^^^sr^rjs^^sm^^^^^-^^ 
Mt«sectag a triangle surfice in tone i-1 nonnal to the triangle surface; 2) the shortest 
vector ftom the point orthogonal to the edge of a triangle surface in frame i-1, and 3) the 
Shortest vector fiom the point to the nearest point in frame i-1. Most often, but not 
always, thzs will be a normal vector, type 1) above. In the example of Figure 41. tninimum 
dt^ance vectors 248 are computed from the points 250 in frame i to the Wangle surfices 
252 of flame i - 1, with the vectors 248 normal to the smfeces 252. 
15 ^' 2 in Figure 40A), three exclusion criteria are appUed to the minimum 

„ _ „^sta„ce_vectors.o£step l.. in order .o-eliminate^,ro«vShwagdiSioiSsTetwi«r^^ - 
frames and to eliminate stray data pomts. First, an minimum distance vectors that relate m 
a boundary element (edge or point) in frame 1 are exchtded. Second, all remaining 
mnnmum distance vectors with an amount exceeding a certain predefined value R. likely 
mdtcatmg stray data points, are excluded, "nuxdly. only Wangle surfaces are talcen into 
consrden^on which form the outside surfice with respect to d.e scamrer viewmg direction. 
Bve^ surfece has by definition two sides. We regard the "outside" surfice as th. surfice of 
the object aiat is oiiented towards the scanner. 

At step 3 (214) the vector sum of aU the minimum distance vectors N , . N » is 

computed. This is shown in Figure 42. with vector 254 representfag the vector sum. 

, At step 4 (215), the median minimal distance vector (t) is eomptrted by multiplying 
the vector sum 254 by the scalar 1/N. Tie median minimal distance vector basicaUy 
consntutes a m^asure of how frame i should be translated in X Y and Z directions it. order 
to better fit to frame i -1. Now, the registration process needs to compute a rotation factor, 
whrch rs explained by steps 5-8. to indicate how frame i needs to be rotated in order to better 
fit frame i-L 

At st^ 5 (216), the X, Y and Z components of the median mimmal distance vector 
xssubtractedfromeverypointinframei. IHs is perfonned by making a copy of the frame i 
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coordinates and operating on the copy as an interim step in the procedinre, the underlying 
data from frame i is unchanged. At the same step the "center of mass" of the points of 
frame i which a^e nbt excluded by step 2 is calculated. The "center of mass" is defined as 
the vector sum of position vectors of aU mentions points scaled by the inverse of the number 
5 ofpoints. 

At step 6 (218) a calculation is made of the cross product of two vectors for every 
point in frame i. With reference to Figure 43, the two vectors are as follows: 1) a position 
vector"^i ^tradingfronf'fhe brigih-JSS of theglobal coordmatesystem'to-the points m 
frame i, subtracted by the vector of the center of mass of the remaining points of frame i as 
10 calculated in step 5, and 2) the identified minimum distance vector Ni for that point. 

At step 7 (220), a calculation is made of the vector sum of the cross vectors 

calculated in step 6, that is the net cross vector X (v^. x rij) for all i points in the frame i, 

J 

where x is tiie cross product operator. 

At step 8 (222), the vector sum of step 7 is weighted against the inverse of the sum 
15 of all squares of the position vectors (Vi) of the points in firame i, to anive at a rotation 
ve^tOT uTuis interpret^as foUowsi^he direction "ofUgives ^ ±ViSm6n^s'^^^^ 
magnitude of U is the angle or amount of rotation. In particular, if we consider Vi to be the 
position vectors from the origin of the coordinate system to the vertex of every point, and Ni 
being the minimal distance vectors defined above, then the weighting is as follows : 

20 U= ^ ^ — 



The reasoning behind this weighting is as follows. If you imagine the distance 
vectors as the realization of linear spring elements, the vector sum of the cross products 
represents the aggregate moment, or rotational discrepancy, generated between both frames. 
25 In the case of smjill deviations between the position of firame i and its final" position, it can 
be assumed that the rotational moment also detemnned the direction of the necessary 
adjustment. The scaling with the help of the inverse of the sum of the squares of the 
position vectors considers the global extension of firame i. That is, the larger the distances 
of the points from the center, the larger is the ratio of rotational moment and angle between 
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the present position and the target position. In a global sense, the mentioned factor (inverse 
of the sum of squares of position vectors) describes this ratio. 

The derivation of the proper scahng factor is by no means an exact calculation. It 
has, however, turned out that using this factor in all empirical cases, the iteration of defining 
overlappmg areas and execution of transformations converges. 

At step 9, the result of step 8 is scaled with an empirical "acceleration factor" f. The 
fectorf s^esjo possibly accelerate this convergence. A value of f of greater than 1 is 
appropnate for -lativeljr-lSge^tadond-^^^^^^ ,,-,3,- 
deteimined empirically. 

At step 10 (226), the result of step 9 is mterpreted as an axis of rotation, the 
magmtude of which indicates the amount by which frame i has to be rotated in order to 
make the local overl^ping areas of both frames lie within each other. Th. magnitude of 
the rotation vector is interpreted as the angle around which frame i has to be rotated. 

A rotation transformation matrix [T] (R) is calculated for frame i This formula 
shows how to convert the rotation vector resulting from step 9, where p is the oridnal 

overl^ping areas of frame i to frame i-l and u is the unit vector of U, , «=ii with " 

\U\ 

components u^>Uy,u.. 

a-cos^«j+cos^ H- cos j3)u^u^-u. sin (l-cos;^« sin^ 
(l-cos^«^«, sin/7 (1-cos^u; -f cos^ (1 -cos^z.^^. sin,^ 
\S^-cos^u^u,-u^sm^ a-cos^«^«^+«^sin>9 {1 - cos jff)ul + cos j3 

To obtain a unique transfomxation operator for calculating the translation and the rotation 
transformation in a closed manner a 4x4 matrix representation is used. The relation between 
the 4x4 representation and the three dimensional rotation represented by 3x3 Matrix [T](R) 
IS as follows 



[^;](i2)= 



0 1; 



and between the 4x4 representation and the three dimensional translation, represerated by 
vector (t) 
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fl 0 0 t, 
0 10?^ 



\Q Q 0 I J 



: To apply, this 4x4 matrices on the three dimensional vectors the following convention is 
made: A three dimensional vector is transformed into the 4 dimensional vector space by 
identifying the. first three components of the 4 vector with the components of tiie three ^ 
5 dimensional vector but the fourth component is always unique. ix,y,zf -^(.x,y,z,l) 

At step 11 (228 in Figure 40 C), a transformation matrix for firame i, [T4](i), is 
calculated by multiplying the rotation matrix [T4] (R) (firom right) by the translation matrix 
[T4](t),from step 4 [rj(0 = [rj(i?) [rj(0 . 

Alternatively, the point cloud of frame i can be separately and independently 
10 operated on by the rotation matrix and the translation vector. 

At step 12 (230), a calculation is made of the square root of the sum of the squares of 
the minimum distance vectors calculated in step 1 (210) of Figure 40A, which indicates the 
closeness factor^uahty of the^sffat^ MA below: ^At-step 12. 

(232) the closeness factor MA is compared to a quahty radex or threshold indicative of a 
15 successful registration (e.g., 50 microns). If the value MA is greater than the quahty index, 
the registration process is repeated another iteration. All of the points m the ith firame are 
updated by operation of the transformation matrix [T4] as indicated at step 234. The 
process goes back to step 1 as indicated at 236, and another iteration is performed of steps 
1-13. 

20 If the closeness factor MA is less than the quaUty index, the registration process 

proceeds for the next frame. As shown in Figure 40D, at step 14 the process retrieves the 
next frame (frame i + 1) and &ame i after appUcation of the transformation matrix [T4] i. 
At step 15 (240), the iterative process of steps 1-14 is repeated for frame i and frame i + 1- 
This process continues until all the N fr^nf^^ave been registered, as iii^cated at step 16 
25 (242). At the end of the process, the result is a set of points comprising the three- 
dimensional model, and a set of transfonnation matrices [T4] ^ to [T4] „ for each of the 
frames (odier than the first fi^e, firame 1), which are used to generate the three 
dimensional model. One frame, such as frame 1, the first one generated, is ctioseix as the 
starting frame for the registration and there is no transformation matrix for that firame. 
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n.dex .s „,et R can be for «™p,e , or „,„,a tes auch a. 1/50 ,™ 
Figure 46 illustrates two wavs in wh.vi, ^ 
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each of the points in frame 1 and frame 2 (after registration to frame 1), and the other steps 
of Figure 40 are performed. As each new frame is retrieved, it is registered to the total sum 
of points from all the previous frames, after the previous registration. 

Figure 48A-48C are a flow diagram of one possible cumulative registration process. 
5 At st^ A (270), the points of the first frame are retrieved from memory. 

At step B , the points of the second frame are retrieved from memory. 
At step C, the transfonnation matrix [T] 2 is retrieved for frame 2. This process 
assumes that the transfonnation matrix for each frame has already been generated, such as 
after a frame to frame regisfration process has been performed. 
10 . At step D, the transformation matrix [T] 2 is applied to frame 2. 

At step E, a registration is performed of the points of frame 1 to the points of frame 
2, after the transformation matrix [T] 2 has been ^plied to frame 2. Basically, the steps l-U 
of Figure 40A-40B are performed. 

At step F, a check is made as to whether the quality of the registration is less than a 
15 threshold. If not, the regisfration is performed again (with the points of frame 2 updated by 
a new transformation mafrix). Steps 278 and 280 are performed over and over again until 
either the quality threshold is met or a maximum number of iterations has occurred. 

If the index is met or the maximum number of iterations has been reached, the 
process proceeds to step G (282). The new fransfonnation matrix for frame 2, designated 
20 [T] 2 ' is obtained and stored. 

At step H, the new transfonnation matrix [T] 2 ' is apphed to the points of frame 2. 
At step I, the new transformed porats of frame 2 are added to a "global contamer". 
The global container is merely memory locations containing the pomts from frame 1 and the 
points of frame 2 as transformed. 
25 At step J, frame 3 and its transfonnation matrix [T] 3 is obtained from memory. 

At step K, the transformation matrix [T] 3 is applied to the points of frame 3. 
At step L, a registration is performed of frame 3, as transformed, to all the pomts m 
the global container...Steps 1-11 of. Figure 40. are perfornied. _ 

At step M, a check is made to see if the quahty index is below the threshold. If not, 
30 another iteration of the regisfration process is performed. This repeats until the quality index 
is below the threshold or a maximum number of iterations is reached. 

If the threshold is met (or the maximum number of iterations is reached), the process 
proceeds to step N. The new transfonnation matrix [T] 3 ' is obtained and stored in memory. 
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At step O, this new transfonnation matrix is ^pUed to the points of frame 3. 

^^/^^P ^' - fi-ne 3 after the transfonnation operation is perfonned are 

added to the global container. 

At step Q, the process of steps A-P of Figure 48 are perfonned for the rest of the N 

^ frames. 

At..epR.a^aeta^f„„„a«o^mattcesm,^..m.■ares,o«do^iheha^ddisk 
of ft. back office server. Tte.e tra^formation n,a«ce.are used whenever ae arisfaed 
^obal coa«r,er (co^piete teee-din,e„.iooa, „ode.) need, to be gen«.ted again a. a Uter 

10 to the raw ftame data comprising femes 2... N. 

^* ^*=P S. the global container is displayed to fl>e user. This can be on the momtor 
of the back office server 28 or on the monitor 20 of the scanning station 16 (Figure 1). 
Smce the global container is a digital representation of the object, i, can be n-ansported 

15 ~ ^ ^ » ~- For example, 

where the back office server has a connection to the toe„,e, the model can be ^ansported 

^so for examples be shared among variou. orthodontic, periodontal or dental speciaHsts so 

to they can collectively study the patient remotely and cooperatively plan care 

Ftgnre 49 ,s an iUusuration of a set of flames, illustrating a variation ftom the 
■ai cumulative registration set forth in Figure 48 r« lr,„ .-^ 

. _ . . ™ P'S^ 49, a different order of flame 

registration is perfonned from that of fVam*. +^ 

F °°^°'»'aat of flame to flame registration, lii particular, frame to 

name registration and cnmnlative n!Bi»ra«,>n „<• u- 
. . ^ ^ uuianve registratirai of Figure 48 are perfoimed in the older in 

Which the flames are obtained, niis neol k. «. , 

. , >»i- ims need not be ae case. In feet, a more accurate 

registration may be obtained by tesistratinn «f «. 

„ . ,. ^ registtation of flames m an order based on 

rfrTT •'""'"''^"■^'^^'^^-'-Stasapaiticularpor.ionof 
an object^uld be legisteied together (or in sequence such they aie registered one after the 

oth^) The order ofiegisMon is mdicated by the left hand colunm of mm*ers. lUe 
nght hand side of Figure 49 merely-iltos.ra.es- that each mine c5ii5rofx7 Y-'and Z 
coordmates for a set of poin.s. me flames need no. and usuaUy will not have the same 

number of points. 

In Figure 49, the registration order is baaed on the location on the surfece of the 
object for a given flame relative u, flie location on tiie surfece for other flames; Fignr. 50 
IS a snnphfied illusion of a set of flames, showing the order in which the frames were 
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obtained, with the neighborliness of the frames relative to other frames being the basis for 
the registration order shown in Figure 49. Frame 7 (F7) is the chosen startmg frame. So, 
the frames surroutnding frame F7 are registered next. This is shown by the order of 
registration being F7, F3, F4, FU, F12, F6, ... as indicated. 
5 Figure 51 is another illustration of a set of frames, with registration of frames 

performed in accordance with the method of Figure 49. The order of registration is indicated 
by the column of numbers to the right of the frames. The maridng m frames 2, 3, 6 and 7 
etc. indicates that those frames have been registered. The maridng is just a way of 
illustrating that the computer keeps hack of which frames have been registered, as a check 
10 to insure that no frames are omitted during the registration procedure of Figure 49 and 50. 
The manner of selecting the other frames, e.g., frame 4 and 5, can be based on a number of 
criteria, such as the order of obtainmg the frame, the neighborhness to other frames already 
registered, and so forth. 

' Figure 52 'is an illustration of cumulative registration based on the first captured 
15 .frame (Fl) as being the base line for all successive regisfrations. This is essentially the 

technique-of-Figure-48. Figure 53.illustrates_an altematLye registration p^oced^^ 

■ each frame in the set of frames is registered to a cumulative registration 3-dimensional 
model of the object, m sequential order, with one iteration of the frame regisfration process. 
This is followed by an updating of the cumulative 3-dimensional model and a repeat of the 
20 registi^tion process with updated values for die transformation matiix [T] for each frame. 
The process continues until , the quahty values are within acceptable Umits, or after a 
predetermined number of iterations have been performed. Still other possibiHties for 
cumulative registration exist. The choice of which one to use will depend on the available 
computing resources, the amount of time required to perform the cumulative registration for 
25 Ihe technique, and the desired accuracy. 

Figure 54 is a screen shot of a workstation computer (e.g., eitiier a scanning station 
or back of&ce server workstation), shoAving the available registration parameters and 
variables that can be changed to optimize the registi^on when performing either- a.frame to 
frame registiration or a cumulative registration. The parameters may vary widely 
30 dependmg on the type of object being scanned, the amount of time needed to obtain a result 
from regisfration, the speed at which the scanner is moved relative to the object and the 
amount of overlap, etc. Figure 54 iUustiates that the user is able to select and modify the 
registiration procedure parameters as they see fit. Two different types of registration are 
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indicate here, a W registration, in which the quality index ("distance limit") is 250 
microns, and a fine registration, wherein the quaUty index is reduced to 50 nncnjns. The 
distance liniit is computed as.the square root of the sum of the squares of the normal vectors 
divided by the number of points in the frame. The tenn "stationary count" indicates the 
number of iterations to continue of little or no improvement in the quaUty index is seen. 
The Radius value refers the filter R shown in Figure 45. The convergence factor 0.10 refers 
to the mmimum amount of improvement needed between successive fiames before a 
stationary count commences. The convergence factor is computed by taking the difference 
of the squares in the quality index of the ith. iteration and the i - 1 th iteration and dividing 
by the square of the quality index of the ith iteration. 

The number of points to register indicates the minimum amount of overlap in points 
(withm boundary R) needed to attempt a registration. An "accelerate"' factor is shown, with 
a value of 1.6. This means that the points are moved in the X, Y and Z directions in the 
transformation matrix by an amount of the net normal vector multipUed by the accelerate 
factor. The use of an accelerate factor has been found to reduce the number of iterations 
. require d to meet the guatity index. — 

. The maximmn iteration count value is a stop value to keep the process from running 
mto an endless loop. The overly size value is a hmit, in terms of mm ^ of the size where 
registration is performed. This serves to screen out stray points from the regisfration 
algonthm. The minimum quota of active points is a minimmn amomit of overlap between 
two frames before registration will be attempted, expressed as a fraction of 1. The 
maxmaum triangle size is a filter to filter out triangle sizes where the size of the triangle is 
too large, indicating a stray data point. The maximal edge length is shnply the maximum 
pemnssible length of one side of one of the triangle surfeces. The Maximal count of 
unsuccessfiil files is the number of unsuccessfirl sequential registrations before a failure of 
the registration process will be declared. . 

Figure 55 is a screen shot from a workstation computer showing a frame to frame 
-regisfration -iii-accordance with Fig^e 40 for two frames in a set of frames. The various 
parameters shown in Figure 54 are selected and used m the frame to frame iteration. In this 
mstance, frame 47 is being registered to frame 46. The surface of frame 46 is shown in 
white, frame 47 is shown m dark tones. The left hand side of Figure 55 shows the results 
of each Iteration, mcluding the nmning time, tixe number of the iteration, the number of 
overlappmg points, the overlap between frames (U), expressed as a fraction of 1 . the qu^ty 
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index MA, and the -value of the filter R. After 3 iterations, the quality index for coarse 
registration was met. The process continued with the fine registration. A series of fine 
registration iterations were performed. Note that the quality index MA improves with each 
registration iteration. 

5 The data from the last twenty iterations, and the final result, of a registration of 

fi^e 2 to fi-ame 1 in a typical scan of teeth are shown in Figure 56. After 45 iterations, the 
distance limit of 30 microns was met (MA = 27.686 microns). Note that the graphical 
representation of flame 1 (white) and firame 2 (darker tones) is such that there is essentially 
an equal amount of firame 1 and firame 2 in the picture. This indicates that a "best fit" 

10 between frames 1 and 2 has been achieved. 

D. Segment Registration 

When scanning any object, such as teeth, the situation may arise in which the 
operator of the scanning cannot capture aU the surfaces of the object in one scanning pass. 
The interruption may be due to the need to physically move the scanner to a location that is 
15 impossible to reach from one location, the need for the patient to take a break firom the 

- - scanning, or some -other reason -When .scanning.teeth, of. ■a_sjngle jaw,_. the scanning is 

typically performed in two or three different segments. First, one side of the jaw is scanned, 
then the front of the jaw, and then the other side of the jaw. In this situation, tiiere are three 
different segments of the object. All the frames of each segment are registered to each 
20 other, typically using a fi^e to frame registration. Then the segments are registered to 
each other. After this has been done, a cumulative registration is performed of the entire 
jaw. 

To perform the segment registration, there must be some way of indicating where at 
least one point in one segment is common ^o another segment. Segment registration thus 
25 requires some overlap between segments. The scanning workstation provides a mechamsm 
to indicate at least one point where two different segments overlap. In the case oi 
scanning of teeth, the operator of the scanner will typicaUy include the canine teeth in scans 
of both sides of the jaw, and in the scan of the front of the teeth. The operator can also be 
instructed to scan these teeth in the. side and front segments. Therefore, the segment 
30 registi-ation proceeds by the user selecting or indicating a point on tiie canine teeth to use for 
performing segment registiration. A procedure referred to herein as "landmarking" is used 
to select the point used to register segments. It wiU be understood that a sinailaif i)rocess 
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After segment registration is performed, a cumulative registration of the entire jaw is 
performed. in accordance with the procedure of Figure 48. After the cumulative 
registration is performed, the virtual three-dimensional model of the entire jaw is presented 
to the orthodontist on the monitor in the back office server workstation 28. 
5 Note that if the scanning is done in one pass, e.g., where it is perfomied on a plaster 

model, there is no need for segment registration. The landmarking step can be eliminated m 
that event, although it may nevertheless be perfomied as a step in placing virtual brackets on 
the teeth objects of the virtual model; 

In planning treatment for the patient, the orthodontist conceptuaHzes teeth as 
10 individual teeth objects that can be moved independently of each other to correct the 
patient's malocclusion. Furthermore, orthodontists are trained to make physical models of 
the patient's dentition from an unpression, cut the teeth from the model, and then 
individually move the teetii relative to each other to provide a target situation which corrects 
for the malocculsion. Therefore the back office server workstation 28 includes interactive 
1 5 treatment planning software which enables the orthodontist to do this with the virtual three- 

dijnensional-model-of-thepatient-'s-dentition.--In-order-to-do-this_treatment.R^^ 

highly desirable therefore to process the three dimensional model resulting from a 
cumulative registration by separating the teeth from the gums and other anatomical 
structure, and presenting the just crowns of the teeth to the orthodontist. This allows virtual 
20 individual teeth objects to be moved independently in three dimensions on the computer. 
This process of separation of the teeth from the cumulative registration into individual teetii 
objectsjw^..be ^described next. 

5 fhe separation process described below has one further advantage, namely requiring 
less membty to represent an individual tooth. Cumulative registration may result m an 
25 extremely, large number of points from a large number of frames to represent any #ven 
tooth. The separation process, as described below, reduces this data set to a single set of 
points that describe a- single surface representing the surface of the tooth. Much less 
, memory, is. requir^. Consequently, the treatment plmning software can process treatment 
planning steps for the teeth more quickly. 
30 ^ 
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The final result, an individual three-dimensional virtual tooth object 312, is tiien 
displayed , to the user, as shown in Figure 59. The result may be displayed on the 
workstation user interface as a three-dimensional superposition of the original data (white) 
. and the separated model of the tooth (darker tones or contrasting color). These tones allow 
5 the user to ascertain whether there is an even distribution of white and dark tones, indicating 
good fit between the scanned tooth 308 and the individual tooth object 312. This step may 
be automated by an algorithm detecting the difference (or the sum of the differences), and 
repeating the process if the difference is too great. 

This process is of course performed for all the teeth. The result is a set of individual 
10 tooth objects for all the teeth in the patient's dentition. The teeth can be displayed either 
alone, or in conjunction wiUi flie surrounding anatomical structures such as shown in Figure 
59. 

Some human interaction is used in the embodiment described above in context of 
Figure 58. While the process could be performed for all the teeth in both arches on the 
15 workstation at the orthodontic clinic, that is not necessary. In particular, since the virtual 

-model- of-the-dentition.and-the. template Jeeth_exist as .digital_data jn_memory, :ttiey can be 

•transported to a remote location and the task of separation of the dentition uito virtual teeth 
objects could be perfomied. at another location. This has the advantage of not tying up the 
back ofiBce workstation or server 28 in the cHnic unduly, and requiring less labor at the 
20 clinic. We therefore contemplate that the function covtld be perfonned as a service of the 
precision appliance service center 26 of Figure 1, or perhaps even by some other entity or 
service provider equipped with the necessary computer hardware and software. Once the 
virtual tooth objects are obtained for aU the teeth in the dentition, the set of virtual tooth 
objects could be sent over the Internet back to the clinic for treatment planning and other 
25 purposes. It would also be possible for the entity performing the separation of tooth objects 
to also present an initial proposed treatment to the orthodontist (such as a target situation, 
location of brackets, and design of orthodontic archwire), and let the orthodontist take the 
process from there or simply indicate her approval. ■ - 

Separation of teeth from the virtual model of the dentition could also be performed 
30 automatically using algorithms to detect incisal edges of the teeth, grooves between teeth, 
and grooves indicating the intersection of the gums and the teeth. 

Two types of errors can occur when separation of teeth objects from other structure 
(e.g., other teeth and gums): 1) the data is selected for a tooth that does not in actuality 
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planes 1000. and 1002. These parts of tooth 308A can be manually erased by the erase 
mode feature described above. 

Anojflier possible method for separation of the teeth, without including extraneous 
anatomical structures, involves allowing the user to cUck with a mouse multiple points on 
5 the surface of the tooth where the tooth intersects adjacent anatomical structures. In Figure 
■ 64B, the user has higWighted areas 1006 on the tooth 308 where the tooth intersects gingival 
. tissue. As- each area is highlighted and selected (e.g., by a cUck of the mouse), the software 
records the coordinates of points associated with these areas 1006. Then, a series of planes 
^ ie contracted %hich connect these points (or surfaces) together. These planes 1008 are 
10 indicated by the hatched area of Figures 64C and 64D. The planes 1008 serve the same 
functions as the cutter planes 1000 and 1002 of Figure 64A; i.e., they define a boundary 
sepai^^g'/the tooth fi-om associated non-tooth anatomical structures, such as gums and 
other teeth. Dependmg on the anatomy of the patient, it may be necessary to highli^t 
closely-spaced areas, as shown in Figure 64B, so that the planes 1008 match the contours of 
15 the gum and tooth. 

Referring- n^w tr. the second problem, the t ooth se paration pr ocess of Figur e 58A -F 

can be forced to use proper data that would otherwise be ignored. Specifically, the user 
cUcks certain areas where original scan data has been wrongfully ignored. Clickmg on the 
area forces the modeling algorithm to pick original data points firom the scan includmg the 
20 selected areas. For example, region 1010 m Figure 64D has scan data associated with it, but 
such data was ignored in a firame to fi:ame registration process. The user highUghts this 
area and points for those areas are filled in fit)m the original scan data for pertinent firames 
covering this area. 

To allow for a safe operation of this user interaction, the modeling algorithm will 
25 mtemaily marie or classify each generated pomt in the vktual tooth model as being based on 
scan data (true points), or if it has been constructed by the algorithm due to the lack of data 
(artificial points, suppUed by the template tooth 310 in Figure 58B). A lack of data wiU 
always occur in the spaces between teeth since the scanner cannot usually capture images of 
the gaps between teeth effectively. A lack of data can also occur due to improper scanning. 
30 The lack of data can be cured to a certaua extent by the modeUng algorithm of Figure 58, 
with the lack of data suppUed by the template tooth, e.g., in the gaps between teeth, and 
adapting this template tooth to the scanned dentition as described above. Artificial pomts 
can be^marked as such and displayed m a different color or using Ughter or darker tones. 
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the software which libraiy of template teeth to use for a new patient based on the relevant 
criteria such as ethnicity. 

Alternatively, in some situations it may be desirable to use a contralateral tooth as 
the template tooth, and not a template tooth from the hbrary of template teeth. In this 
5 situation, a tooth object of one of the contralateral teeth is obtamed from a template tooth in 
■ a Ubrary and the cumulative registration scan. Then, the tooth object of the contralateral 
tooth is obtained by using the contralateral tooth as the template tooth and the cumulative 
registration scan as explained above. 

In some situations, the template tooth may need to be modified. A few example of 
10 such situations are a partially enJ5)ted molar, a tootii that has been chipped, or a tooth that 
has been the subject of extensive previous dental work. Since the template tooth exists as a 
mathematical model, it can be displayed on the user interface and modified. The 
modification can be made using suitable navigation software and a cUpping or erasing 
feature to delete part of the model. One way is providing a clipping plane feature, by which 
15 a plane intersects the template tooth in a orientation and location defined by the user usmg 

suitable navigation- tools ^The portion of„the_teniplatetootii on, one sid e of Pj^^J^ 

deleted. The user positions the plane at the desired location on tiie template tootii to 
roughly match the anatomical stmcture of the tooth in question. This process will result m a 
smooth execution of the tooth separation algorithm and result in a virtual tooth model that 
20 substantially exactly matches the structure of the patient's tooth. 

The virtual tooth model may be extended beyond merely the crowns of the teeth. 
For example, a Ubrary of standardized virtual root templates for each of the teeth may be 
stored in the memory of the workstation. As individual virtual models of each tooth are 
created, the standardized root for that tootii are matched up with the virtual tooth model to 
25 thereby created an individual virtual tooth model of the entire tooth. 

This process can be extended to templates of virtual gum tissue. On one hand, after 
separation of tiie individual virtual tootii models from tiie gum tissue tiie remaining portion 
of the scan data depicts tiie gum tissue (or at least aportion of tiie gum tissue, depending on 
the tissue scanned). This gum tissue may be substantially incomplete. The incomplete 
30 portions can be suppUed by a template of virtual gum tissue, e.g., gums for an . entire arch. 
The template of virtual gum tissue can be scaled up or down or modified as may be 
necessary to fit tiie anatomical sti^cture of tiie patient. A registiation of tiie template gum 
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taken of the gums and associated bony alveolar structures and these anatomical structures 
are cast in plastic or wax. Pre-fonned teeth are set into the wax in a desired occlusion. The 
dentures are cast in acryUc using the lost wax technique. This process can be automated 
using the scanning methods described hereia and using virmal three-dimensional template 
5 teeth. First, the gums and associated anatomical structures are scaimed and represented as a 
three-dimensional virtual model on the workstation. Then, virtual template teeth are 
retrieved from memory. The template teeth are sized up or down as necessary to conform to 
the archfonn represented by the virtual model of the gums. The virtual template teeth are 
then placed on the archfoim. At this point, a three-dunensional virtual model of the teeth, 
10 gums and associated anatomical structures is represented in the workstation memory as a 
three-dimensional virtual object. This digital object can be exported anywhere, such as to a 
remote location where dentures are manufactured. From this object, a denture can be 
manufactured from a variety of techniques, including milling and casting. For example, a 
stereoUthographic physical model of the dentition and/or gums can be made and a denture 
1 5 cast in a mold obtained from the physical model using the lost wax technique. 

The-virtual template teeth can- also be-used-in-forensic-dentistiy,.i.e.,. reconstruction 

of the identity of a victim from teeth. As an example, a jaw containing some teeth can be 
scanned as described above and represented as a three-dimensional virtual object. Missmg 
teeth can be reconstructed by inq>orting virtual template teeth and placing them on the 
20 virtual object. The virtual template teeth may be based on age or ethnicity if such 
information is known. Contra-lateral teeth can be constructed by using existing seamed 
teeth as the template tooth and placing the scanned tooth in the contralateral position. 
Eventually, a complete virtual representation of ttie dentition can be obtained and viewed on 
the workstation. The shape of the face of the victim can be reconstructed by adding 
25 template virtual objects comprising soft tissue, gums, lips, cheeks, skm, hair, etc., and 
modifying the template objects using navigational tools based on the three-dimensional 
object or other information known about the victim. 

Another example of using template teeth is for purposes of diagnosis and detection 
of tooth wearing, e.g., due to bruxism. In this example, the original scan taken of the 
30 patient is converted into a three-dimensional virtual model. The individual teeth are 
optically separated into virtual three-dimensional tooth objects as described above. Either 
this original virtual model of the entire dentition or the set of virtual three-dimensional tooth 
objects can be considered as a template. Over the course of time, the dentition is scanned 
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1) navigating throu^ the treatment planning software until the virtual model of the 
dentition and the virtual brackets are displayed (this can be either the target 
situation or the maloccliision); 

2) selecting a bracket for movement by either cHcking on the bracket or selecting a 
5 bracket number from a drop-down menu; 

3) accessing navigational controls for the bracket, such as by cUcking on an icon 
that displays navigational controls for moving virtual objects such as brackets; 

4) allowmg the user to select either move the teeth with the bracket or move the 
bracket freely in three dimensions; and 

10 5) using the navigational controls to move the brackets in three dimensions as 

desired. 

If the bracket is moved independent of the tooth model, when the user is finished with the 
movement of the bracket the virtual tooth is moved to the location of the bracket Bonding 
corrections for bonding the bracket to the tooth are updated. The bracket is iJien virtually 
15 bonded to the tooth. This process can be perfonmed for each tooth. The result is that the 

- ■ orthodontist customized the placement of vhtual brackets-to.the .teeth-. _The .^chwire, wMch _ 

passes through the bracket slots, will have the required bends to move the teeth to the 
desired target situation regardless of the positioning of the brackets on the teeth. 

The combination of the displayed set of virtual orthodontic brackets, together with 
20 the virtual orfhodontic archwire, thus presents to the user a customized virtual orthodontic 
apphance. The virtue of the customized virtual orthodontic ^pUance is that it can be 
studied, modified, shared between two computers, and transported electronically over a 
communications medium for fabrication of the orthodontic appliance. The treatment 
planning software is essentially a speciahzed CAD/CAM system that allows the design of 
25 vuto^ly any configuration of tooth objects, bracket objects, wire objects and otter 
appUances and objects. Because these objects exist as independent mathematical objects, 
they can be selectively displayed together or alone. For example, the treatment planning 
software displays an icon or button on the user interface that allows the user to select or 
deselect the teeth, wires, brackets or virtual objects or appHances, as desired. For example, 
30 the teeth and archwire can be displayed together with the brackets deleted from the user 
interface. The orthodontist can then select an individual tooth object, move it in three 
dimensions, and the movement of the tooth carried over to a repositioning of the bracket m 
three dimensions and a changing of the shape of fee archwire. 
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encompass other types of appliances, such as an approach based on customized brackets, 
retainers, or the removable ahgning devices mentioned earUer. In a bracket embodiment, 
the memory contains a Ubrary of virtual, three-dimensional orthodontic brackets. The 
software permits a user to access the virtual brackets through a suitable screen display, and 
5. place the virtual brackets on the virtual model of the dentition of the patient. This bracket 
bonding position can be customized on a tooth by tooth basis to suit individual patient 
anatomy. Because the tooth models, brackets and archwire are individual objects, and 
stored as such in memory, the treatment plamiing apparatus can simultaneously display the 
virtual brackets, the archwire and the virtual model of the dentition, or some lesser 
10 combination, such as just the brackets, just the dentition, or the brackets and the archwire 
but not the teeth. The same holds true with other appUance systems. 

hi a preferred embodiment, the virtual model of teeth comprises a set of virtual, 
individual three-dimensional tooth objects. A method of obtaining the tooth objects fiom a 
scan of teeth, and obtaining other virtual objects of associated anatomical structures, e.g., 
15 gums, roots and bone is described. When the teeth are separated from each other and ftom 

^3ieguins,-they-canbeindividuaHy-manipulated.--Tli^ 

individually selected and moved relative to other teeth in the set of virtual tooth objects. 
This feature permits individual, customized tooth positioning on a tooth by tooth basis. 
These positioning can be in terms or angular rotation about three axis, or translation m 
20 transverse, sagittal or coronal planes. Additionally, various measurement features are 
provided for quantifying the amount of movement. 

:One of fee primary tools in the treatment planning apparatus is the selection and 
customization or a desired or target archform. Again, because the teeth are individual tooih 
objects, they can be moved independently of each other to define an ideal arch. This 
25 .. development of the targefc archform could be calculated using interpolation or cubic spline 
algorithms. Alternatively, it can be customized by the user specifying a type of archfonn 
(e.g. Roth), and the tooth are moved onto that archform or some modification of that 
- aichfprm. , The archform can be sh^ed to meet the anatomical constraints of-the patient. 
Afta: the initial 'archform is designed, the user can again position the teeth on the archform 
30 as they deem appropriate on a tooth by tooth basis. The treatment planning software thus 
enables the movement of the virtual tooth objects onto an archform which may represent, at 
least in part, a proposed treatment objective for the patient. 



83 



wo 01/80761 

PCT/US01/1J969 

inCudin!"' ^"^^ "° ""^^'^ '^^ « -ftware. 

tt^po x.onof the v«.al b.ck.ts *e .==0, witt r«p,c. to each other. oropp^ingLh 
add.t-onaIco.ec.on.. Bonding co,^«on, ^ ^ J^,^ 

Planninrr ^ ^ 

. ™1 : "'"^ ^ ^ th^e-dhnensiona, virtua, .ode. of teeth 

Ta JlT ° " "''^ '° -"-^ ^ .eeth of .ha via, .od.. 

rloTl r ^ ^ -ved u. the targe. 

Tt^ot " "-^^ °^ » -"-i- - .ove the teeth to the targe, 

diapla™'^^ ^'"^ of providing screen 

1^ onTT '^^"■^ or«.odontic appHaaces. e.g.. 

br^^Lt^""'^'"''^^^""^''- Ah^raryofBrevirtlhrac^can 

L^lr? ^"■^^^^°'"'^-^°--^'««-=™eo.ofthcbracfcc.fto=t 
^glZT <o a new position. Accotdingiy, a.e software provides 

navgahonaUools ^.hhng a „^ to change p^^i^on of the brackets revive to Z t^ti. 

movedri!^"^"' ^ - on individual tooth objects which can be 

ouect user specified movement mrf i,„ .jj- . 

aoBliance ^.1 • v ^ ""j^"' comprising an orthodontic 

Changed m «^e dmtens.ons to move me teeth. Alten,a«vely. an archwire shape can be 



84 



wo 01/80761 PCT/US01/n969 

defined which fits into the slots i the brackets. Movement of the archwire can be simulated, 
resulting ui a simulation; of tooth movement. 

The treatment planning software includes features enabling more accurate diagnosis. 
For one thing, die virtixal model of the dentition can be manipulated in three dimensions at 

5 wm, resulting in complete visual assessment of the model. Measurement tools are also 
provided by which the orthodontist can determine die distance between any two points on 
the model. This allows the user to quantify the patient's morphology both at initial and at 
target stages. Thus, treatment progress, proposed changes in appliance design, or tooth 
movement can be quantified precisely. By measuring the differences and changes in 

10 morphology during the care cycle, the orthodontist can quickly and accurately assess patient 
treatment. Changes in treatment can be made early on. The result is shorter treatment times 
(and die abihty for the orthodontist to service more patients per year). 

The treatment planning system incorporates virtual objects cpmpiising orthodontic 
appUances that may be used to treat the patient. The invention provides for desi^ of 

15 complete appliance systems and simulation of various treatment scenarios prior to initiation 

- - - of "treatment 



A bite registration scan is obtained firom die patient to spatially correlate die scans 
of the upper and lower jaws when tiie dentition is clenched. This scan is used to provide a 
registration of die upper and lower jaw to detemmie flie correct relative position. This bite 
20 registration scan is usually needed at the beginning of treatment to set the relation between 
the upper and lower jaws. 

Landmarks such as shown in Figure 57 are dien placed on the labial surfaces of all 
the teedi. The illustrated embodnnent places landmarks manuaUy, but this process could be 
automated. The landmarks are placed initiaUy on the molars and the front teeth, and an 
25 estimated position for die landmarks on die odier teeth can be made, such as in the same 
plane, based on die relative position of die landmark widi respect to die gingival tissue and 
incisal edge of the tooth, or other factors. 

The landmarks are placed at die location where die orthodontist expects to -place an 
orthodontic bracket to correct die malocclusion. The bracket shape is shown on die 
monitor 30 (Figure 1). Three-dimensional templates for a variety of commerciaUy 
available brackets are stored in memory and die software asks die orthodontist to select a 
particularmanufacturerandstyleofbrackettousewittitiiepatieht. Thus, as die landmarks 
are placed, virtual brackets appear in die computer model on die labial surfaces of the teeth 
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312 position, on a tooth by tooth basis to thereby optimize treatment planning for the 
patient.- • 

The result- of the treatment planning is the generation of a set of bracket placement 
positions and the display on the monitor of the shape of a customized orthodontic archwire 
5 to treat the malocclusion. Information as to the location of the brackets, the three- 
dimensional model of the nialocclusion, the three dimensional model of the target situation, 
and the type of archwire to use are sent to the precision appliance center 26 of Figure 1 . A 
customized oiihodontic archwire is manufactiired in accordance with the bracket location 
and type and the target Situation for the patient. Additionally, a transfer tray is 
10 manufa.ctiired to assist the orthodontist to place the brackets at the proper location. The 
transfer tray, brackets and archwire are shipped to the orthodontist's clinic. 22. The 
orthodontic appliance is then applied to the patient and treatmoat commences. 

Because the hand-held scanner, allows for scans of the dentition in a matter of 
minutes, the scanner becomes an important tool in monitoring treatment. As the treatment 
15 progresses, tiie movement and position of the teeth during treatment can be quantified with 

ghigh degree of pr^^i'^inri The orthodontist ca n discern durin g treatment that correc tions 

in the wire need to be made, for example due to biological influences affecting tooth 
movement The treatment planning software on tiie workstation displays the cuirent 
sitiiation, and also the target sitiiation. A new customized archwire is designed on the 
20 computer. The relevant information for making the new archwire is sent to tiie precision 
appliance service center and a new archwire is manufactiired and shipped to the clinic. 

Monitoring scans are taken during tieatinent to measure and quantify progress and 
detect deviations from the expected tireahnent. Since each of the tooth objects is already 
stored, the monitoring scan need not be of the entire dentition, but ratiier needs to only be of 
25 one surface, such as the occlusal surface, or the lingual surface, or some combination of the 
two. A bite scan with the teetii in a clenched condition is also taken to get tiie current upper 
and lower relation. The position of the rest of tiie teetii is obtained from tiie virtiial tootii 
objects 312 of the patient's teetii (Figure 58F). After flie monitoring scans-are performed, a 
registration is performed of tiie scamied data relative to tiie tootii models to complete tiie 
30 teetii and derive a cuirent virtual model of tiie patient at tiiat point in tireatinent. Stiidy of 
tiiis virtual model and comparison to tiie target virtual model of tiie teetii and tiie virtual 
model at tiie beginning of tireatinent can indicate whetiier progress is as anticipated or if 
additional correction to the orfliodontic ^pUance need to be made. These corrections wiU 
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the orthodontist to select a particular manufacturer and style of bracket to use with the 
patient. Thus, as the landmarks 302 are placed, virtual brackets appear in the computer 
model on the labial surfaces of the teeth where the orthodontist desires to place the brackets. 
The orthodontist can move the bracket position depending on the type of forces the 

5 orthodontist wishes to create on teeth to correct the malocclusion. Because the brackets are 
individual objects and stored in memory, when they are placed on the surface of virtual 
teeth complete position information is known in three dimensions. As such, the brackets can 
be displayed either alone, or in conjunction with teeth, or hidden from view, by means of 
appropriate user specified commands on the user interface. For example, the screen display 

10 showing the target or current stage can have an icon indicating hide brackets, or display 
brackets, and activating the icon causes the brackets to be hid or displayed. The same is true 
for other virtual objects that exist independently of other objects, such as tooth models and 
the archwire. 

With the teeth now separated into individual tooth objects, and the orthodontist can 
15 now view the current target stage, custom dfesign a target situation for the patient, and 
- -design-the appliance to-treat the- malocclusion.- -Jhese-.aspects__wiUjtiow be de^^ 

further detail. 



20 



Viewing the observed (current') stage 



Figure 69 is a screen shot showing a three-dimensional model- 18 of a malocclusion, 
showing the teeth 312 in both the upper and lower arches 326 and 328, respectively. The 
screen 330 includes a row of icons 332 across the upper portion of the display, which are 
associated with various tools available to the user to view the dentition, virtual brackets, and 
25 current and target archforms. The lower portion 334 of the screen includes a set of tabs 336 
that are accessed in various aspects of treatment planning. These tabs 336 include a patient 
tab 338, which accesses the screen of Figure 66. A limits tab 340 allows a user to 
breakdown the tooth movement between observed and target stages into stages, such as 30 
percent, 50 percent and 75 percent, and display the tooth positions for both arches at these 
30 positions. A differences tab 342 quantifies the differences (in tenns of translation and 
rotation) between the observed and target stages for each tooth. The space management tab 
344 permits the user to simulate extraction of one or more teeth and adjust the spacmg 
between teeth in either arch. A bonding conrection tab 346 allows for adjustment of tooth 
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with the teeth. Furtiier, the camera navigational tools allow the user to zoom in an zoom 
out in any desired degree. However, the virtual teeth 312 and virtual brackets 400 are 
individual three-dimensional objects which can be selected and moved independently. One 
way of moving objects is by entering new positional values (e.g, in terms of mm of 
5 displacement or angle of rotation, as described later). Another method provided by the 
software is using object navigational controls, activated by clicking the icon 353 or by 
accessing the fimction via a tools menu. The object navigation controls allow the user to 
move the object based on orthodontic judgment and visual feedback. The amount of 
movement is stored and can be displayed using numerical position information. As will be 
10 discussed iii further detail below, the bracket position can be individually adjusted on a 
tooth by tooth basis. Furthermore, the camera navigation icons pamit navigation of the 
archforms (i.e., the teeth placed on some selected archform), navigation of the brackets, or 
navigation of the archwire. 

The object navigation tools first require an object (e.g., tooth, bracket, archwire, etc.) 
15 to be selected. To select and deselect any object displayed on the screen, the user places the 
... cursor over the obj^^t and double clicks the mouse. The selected object is highlighted m a 
separate color. Additional objects are selected by pressing and holding down the <CTRL> 
button while double clicking additional objects. To magnify the object, the object is 
selected as described above ..and the icon 341 is cUcked. 
20 To move the selected objects, the software provides an object navigation icon 353. 

When the icon 353 is selected, object navigational tools appear on Hhc screen 330. These 
navigational tools 355 are shown in Figure 68 and Figure 69. The object navigational tools 
355 comprise a large circle 357, a small circle 359 and a smaU rectangle 361. First, the 
object is selected as described above. Then, the object navigation icon 353 is clicked, 
25 activating the tools 355 such that they are displayed. The user then positions the mouse 
pointer relative to the tools 355 and presses and drags as described below to position the 
object. When the mouse pointer is outside the large circle 357, when they start dragging 
the object is turned either clockwise or counterclockwise depending- on the direction of 
dragging. When the mouse pointer is positioned within the large circle 357, when they start 
30 dragging they rotate the object in any direction. When they start dragging from inside the 
small circle 359, the object is moved in one plane. When they start dragging from inside 
the rectangle 361, by dragging down the object is moved closer, by dragging upward the 
object is moved farther away. 
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Clockwise or counterclockwise. Then they start dragging inside the large circle 357, the 
plane is rotated in the direction indicated by the dragging. When they start dragging from 
inside the small circle 359, the cUpping plane is moved in the direction of the dragging. 
When they start from inside the rectangle 361, if they drag up they cut less into the model, 
5 by dragging down they cut further iato the model. 

Viewing and adjusting initial bit e registration 

The first step in typical treatment planning is deciding where to place the teeth in 
three-dimensional space. This will ordinarily involve a definition or fixation of the 
10 vertical level of the teeth relative to the bones, and defining an occlusal plane, and 
adjusting the occlusal plane sagittally and transversely. This, in turn, will ordinarily 
involves an assessment of structural relationship between the teeth and the maxilla and 
mandible. The orthodontist perfonns this by accessing and studying X-ray, CAT scan, 
photographs or other two dimensional data stored in the patient records portion of the 
15 treatment planning software, and of course the three-dimensional model of the 

malocclusionr-witii- the-teeth-either-represented-as- individual. to.oth_Qbj eels .ox.in£0^^ 

with the suirounding anatomical tissue. The mid-sagittal profile of the incisors and 
molars is set up by superimposing the mid-sagittal plane of the teeth over the X-ray. 

Figure 66 is a screen shot firom the workstation rmming the treatment planning 
20 software showing a patient information screen. The screen includes a region 500 for 
storing various photographs 502 of the patient's head and face, and various views of the 
patients dentition. The photographs are taken with a digital camera and loaded into the 
woricstation, and accessed from the treatment planning software. The patient information 
section of the software also includes separate screens for entering and displaying other 
25 pertinent information for the patient, accessed through a menu 504. These additional 
screens (not shown) include the patient demographics, patient medical, dental and 
orthodontic history, examination notes, and X-rays. 

To assist this process, the treatment planning software provides the ability to view 
and adjust the initial bite registration. The initial bite registration between the upper and 
30 lower arches can be modified using the U/L relation tab 327 of Figure 69. The user can 
move or rotate the lower jaw relative to the upper jaw by entering values in the field 366. 
The user can also simulate a chewing motion of the upper and lower jaws by moving the 
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this wire is flat. . It is also possible to design the wire to adapt to the malocclusion in 
vertical direction to create a Curve of Spee "if desired. The geometry in the transverse 
direction can also be changed, such as the transverse curve of Monson estabUshing an 
inclination of the teeth in Ihe coronal plane. 

5 Figure 76 is a screen shot ftom an orthodontic workstation showing the computer 

model of the patient's teeth objects 312 positioned in a target or desired condition. The 
illustration is the result of the user selecting an archform for the patient &om a known type 
of archform and the computer placing the virtual brackets along the arch selected by the 
user. This is executed by placing or threading the virtual brackets along the archform or 

10 curve selected by the orthodontist. The brackets are omitted from Figure 76, but are shown 
in Figure 77. When the initial target archform is created, the sUde line tab shown in Figure 
67 is activated. 

The initial target archform presented to the user in Figure 76 is only an mitial 
archform. The treatment planning software allows the orthodontist to change many variables 
15 in the target situation, simply by entering new values in the sUde line area 402 of the screen 

di^l^FigiiF^rp-shows-some^ - - 

shape of the arch, including the distance between the cuspids, the distance between the rear- 
most molars, the center line offset, and the radius of curvature at the front of the teeth. 
Slide line area also permits the user to select a symmetrical archform or an asymmetncal 
20 archform, and apply corrections on the right and left quadrants as indicated. As values are 
entered in area 402, the shape of the archform is instantaneously modified on the screen 
display, alloAving the user to simulate various potential archform configurations for the 
patient. • ■ : t 

" :ih geheratirig the archforms shown in Figures 76 and 77, the user wiU ordinarily set 
up-the lower arch first. . The upper arch is automatically derived from the lower arch. The 
view the arch forms in several ways using three arch icons 390, 392 and 394. The 
arch icon 390 is for the malocclusion arch, which causes a blue line to appear on the screen 
which exhibits the curvature of the observed arch or malocclusiiiii:" The line'passes through 
■ tlie slote on the virtual brackets, as placed on the teeth. The arch icon 392 is for the target 
30 arch, which represents a custom archwire passing through the bracket slots of the teeth m 
the target situation. The line 396 m Figure 76 represents this arch. The arch icon 394 if 
for an ideal spline or curve generated by the software to represent an optimal shape. The 
arch retains a parabohc shape as the user manipulates the target arch usmg the entries m the 
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measurement,tpol described earlier. Bracket manufacturers have recommendations for this 
distance. If the initial landmarking has placed the brackets outside of the recommended 
distance, this distance can be adjusted by using the object navigational tools. Alternatively, 
the user can select preferred values for the archforai and the bracket set (bracket 

5 manufacture, bracket technique and wire type, e.g., straight) and the landmarks and virtual 
brackets will be placed on the teeth at the recommended distance. 

The bracket placement can also be perfomied interactively by the user. The user 
looks at every tooth 312 one by one (using a screen such as the screen shot of Figure 75) to 
see if they are basically satisfied vvith the bracket 400 position, e.g., angulation, side to side 

10 position, and its relation to teeth in the opposing jaw. The user may be performing this step 
while viewing the malocclusion, or the target stage. To improve the position of the virtual 
bracket 400, the user selects the bracket, zooms in if necessary, and adjusts the position of 
the bracket on the tooth surface using the navigational tools as described earHer. 

15 rorrection nf Tndividual Tooth Pos ition In Target Archform 

- iOt^t&earcBiom hasbeen-^^^ placement- optimized, the.user . „ 

can adjust the individual tooth position on a tooth by tooth basis in the target arch. There 
are three ways this can be performed. First, the user can use the tables in the Target 
Correction tab. See, for example. Figure 78, in which the user has entered a value of - 15 
20 degrees for rotation of tooth number 41, and the tooth is rotated by that amount. The 
correction is reaUzed by a bend in the archwire 396. The bracket position on the tooth does 
not change in this example. The target corrections tab 410 permits any values to be entered 
for mesial, buccal and coronal translation in three planes of space, and torque, angulation 
and rotation movements about three orthogonal axis. Thus, independent tooth position 
25 corrections are available in 6 degrees of freedom for every tooth, merely by entering values 
in the tables in the target corrections tab 410. Another example is shown in Figure 81, m 
which a new target position is shown for tooth number 41 . 

Secondly, the user can move teeth using the Bonding Corrections tab"346 of Figure 
69. An example is shown in Figure 85. The bonding corrections tab 346 allow the user to 
30 enter new values for tooth position for any tooth in the arch by merely entering values m the 
appropriate cell. The selected tooth 312A (tooth number 41) is moved as indicated (here, a 
new rotatiori value of -15 degrees is entered). The virtual bracket remains in the same 
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includes a section 430 where the user can view the distance in mm in which the wire will 
slide relative to the brackets, on a tooth by tooth basis, when the teeth are moved from the 
current situation . to the target situation. Note that a colHsion is detected for tooth 22 in 
movement of. the. tooth from the current situation to the target situation. This can be 
5 resolved in several possible ways in accordance with the teachings of U.S. Patent 
■ ' ; . ■: to Sachdeva et serial no. 09/451,609 aUowed December 7, 2000, the 
contents of which are incorporated by reference herein. 

. Additional Wire:Bending Corrections 

10 The wire offeets tab 426 (see Figure 73 and Figure 87) allows the user to simulate 

bending a wire 396 to the corrected tooth position while simultaneously retaining the 
original; bracket position. Note that in Figure 87 the wire 396 is flat with no corrections 
indicated. The user highlights one of the teeth in the virtual model and enters new values 
for tooth position. The change is reflected in a new shape for the wire 396, The tab also 

15 allows the user to build in over/under compensation into the archwire. These settings do 

not-affect the-bracket pbsitionr^e characteristies of-the wire bends,-such-as-the.r^^ 

curvature, can be controlled by accessmg a shdings tab (not shown). Another tab, forces 
428, displays approximations of the forces the wire applies to each tooth to displace it from 
its initial position to the target position. 

20 

Treatment Stages 

Since the difference between the current situation and the target situation is 
quantifiable in terms of millimeters of movement or degrees of rotation about three axes, the 
treatment for the patient can be broken up into segments or stages with each stage defined 
25 aibitraiily. For example, the orthodontist can specify a first treatment stage as the 
movement of the teeth from the initial position half-way to the final position. The software 
includes a screen that permits the user to view die position of the teeth when moved to the 
half-way position- Basically, the simulation sunply multiplies the differences in" tooth 
position between initial and target stages by 0.5 and the resulting tooth positions are 
30 displayed W the workstation. Additionally, the user can specify further intermediate 
positions, such as one fourth or three fourths. With this feature, the orthodontist can 
monitor treatmeiit and compare the progress of the treatment with the limits or stages that 
have been .set When the patient comes in for a visit during treatment, the patient s 
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the user to view the change in position that has occurred. The measurement marker features 
described earlier allow the user to quantify precisely the amount of movement. 

Any deviations between the therapeutic result that is observed and the expected 
result can be captured precisely and at an early stage in treatment using the scanning and 
treatment planning features described herein, and corrected for. For example, the 
orthodontist may need to place additional bends in the archwire. Such additional bends can 
be performed by simulating the wire shape on the screen, displaymg the wire only on the 
screen, and printing out the screen and using it as a template for bending the wire. The 
current situation could also be forwarded to the precision appUance center for manufacture 
of a new appUance. Of course, these monitoring and treatment corrections are applicable to 
any type of appliance selected for the patient. 



Parts.-. Appliance Manufacturing 
15 The appliance that is manufactured in accordance with the treatment planned for tte 

patient-can vaiy-within- the-scope-of-the-invention-and -will- <iepend-on-the_appUcation,. _ _ 

including the type of appUance desired by the orthodontist and the patient. Obviously, the 
treatment planning and simulation features described above can be used in wide variety of 
treatment regimes, including flat wires and brackets, finishing wires, retainers. Herbst 
20 devices, expansion devices, and removable, transparent aUgning devices such as those 
furnished by AUgn Technologies. For example, the movement of the teeth firom the cuirent 
or observed stage to the target or treatment stage can be broken into a series of movement 
steps. For each step in the process, the position of the teeth in that stage is known by virtue 
of the manipulation of the individual tooth models in the treatment planning software. 
25 These tooth positions can be used to manufacture the aUgning devices. 

In a representative embodiment, the results of the treatment planning software are 
used to generate a customized orthodontic archwire and a bracket placement tiay for precise 
placement of off-the-shelf brackets onto the patient's teeth.' When tiie tteatinent" planning 
featiires have been executed to the satisfaction of the orthodontist and the proposed target 
30 sitiiation finaUzed, the treatinent planning software will store the following information (m 
addition to the patient records): 

1) the virtual model of the current stage or malocclusion; 
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BasicaUy, the position of the bracket slots a«H .1, ^, 
teetii^ein a target situation i ■ ^ ''^^^ets, when the 

treatment planniTg s^ ^^^^^ 

B wxtware. This position ni^iu^ u ,_ . , 
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(e.. the .ength) iLcT^ kn^lrr ^"^^ ^ ^ 

to derive a three-dimensional set of li„ ^^^P^' P--^^^ 

0 passing through the bracket slots " th ^ '^^^^ represent the shape of an arch we 
bends that comiect the bracket slo" tog^^ T'"- '^'""'"""^ "^""^ 
bends are fed as an input ftle to a wir! beldin^Trr "^V^^^ ^""^^ 
Wthewiresi., the shape and size of the ^fs td«ir T ' ^^'^ ""'^ ^^'^ 

situation. From this information robot " "^"^^ ^ *^Set 

' thedesiredshape. ' ^--ted to b^d the archwire into 

The bracket placement tray is senarat.K, 
other similar technique The treatL t T "^^^ stereoHthography or 

-and- superimposes the brackets on'&e '^^"-- ^^^^ 

comprising the three-dimensional tnnth u- ° ^ three-dimensional model 

-o. in .e „.e„ed rrj^ ^'""^ ^ 

to a StereoHthography (SLA) in^rument Tie Sr ^""^ -""el i. supplied 

Of the teeu. wi«. the ht^ts snpeHtnposed I^etT^ ATr^"^""^ ^"'""^ 
above the SLA model and the model »rf , ■, thermoplastic foU is placed 

the model and Ware placed witUn a prcssnre chamber. "tte 
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chamber is pressurized so that the foil envelops the dentition and the brackets. After 
cooling, the foil is removed from the model.. The foil, now in the shape of a transfer tray, 
has small indentations where the brackets are located. Real brackets are placed in these 
indentations. The orthodontist uses indirect bonding techniques to bond the brackets to the 

5 teeth. The transfer tray positions the brackets on the teeth at the desired location. After the 
bonding is complete, the orthodontist removes the transfer tray, resulting in the brackets 
bonded to the teeth at the desired location. A fiirtfaer scan of the dentition can be made at 
this step to verify the position of the brackets. Any substantial deviation in the bracket 
position can be accounted for by modification of the archwire, again using the treatment 

10 planning features described above. 

There will always be some small gap between the bracket base (the part bonded to 
the tooth) and the tooth, as an off-the-shelf bracket wiU never precisely match the individual 
tooth of any given patient. One option is to fill the gap using a surplus of bonding adhesive 
during bonding. Another option is to equip the base of the bracket with a customized pad 

15 made from an adhesive. 

_ . —— Customized- bracket pads-cari-be manufaetured-using a-variety^of-techniques,_ One_ 

possibiUty is to bond a blob of adhesive to a bracket base, and mill the blob using a miUmg 
machine to match the tooth surface. Since the tooth surface is knovm precisely from the 
scanning, and the position of the bracket base relative to the surface is also known precisely, 
20 a very accurate bracket pad can be manufectured in this fashion. Another technique is to 
stamp out a bracket pad using stamping machine either in the desired shape or as a blank 
and miUing the stamped pad to the desired shape. A third possibiHty is creating a negative 
of the customized pad, forming the pad in a mold, bonding the pad to the bracket and the 
orthodontist trimming the pad as necessary when the pad is bonded to the tooth. 
25 Once the brackets have been bonded to the teeth, a scan of the bracket placement is 

made. The scan is compared to the digital template of the expected bracket position. If the 
bracket is placed mcorrectly, the bracket can be re-bonded to the tooth. Alternatively, 
corrections to the wire may be made if necessary to account for dispraceiheiit of the brackets 
from their expected position. Basically, this is done by simulating the position of the teeth 
30 with the actual bracket placement at the target situation and correcting the shape of the 
archwire as necessary, or obtaining a new archwire based on the actual placement of the 
brackets. 
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it IS also possible to manufacture H., 
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5 including features to hold die bractetl d f ^^""^ "'^''^ sophisticated geometry 

dentition. Additionally, bracket placelTt iir^'T "^'^^^^ '""''^^ 
models of the dentition and the b ^"^"^ ^""^ three-dimensional 

Andreiko patent cited earlier. '"^'"^'''^ ^ ^^^P^^' 

As another possibility, the customized bracket bonH' . 
10 tooth/bracket geometry can be fabricated '^'^^ on the virtual 
material which conforms to the surface of the tol t ' """^ 
anow the bracket bonding pad to b • ^ ^ Placement jig for the toodi to 

bonded to fixe bracket bondin. pad l^"""'^ ^^'^^^^^ tooth. n.en the bracket is 

^ ---ed^sXitr:™^^^ 

5 P^-»^gfi-ctionscanbeperfonnedwithfl.eL 7^ ^""'"""'"^ "^"^ 

J^e_^^p,^^ed.embodiment-u^^^^^^^ ^''^^^ — • 

. ^eld scanner, this is not necessary or re^uTer ^''^''^'""^^^^^ 
for common types of files of 3D objects includmg ^^^^^2"' ""^^^ 
Additionally, another key aspect of th J ' 
> the placing of brackets onto tooth models " ^ '"^ ^^""^^ thatit pennits 

those brackets to the teeth so that they ' ^^^"^ "^^^ 'l^onding" 

described a landmaxking feature by whThr^'"*" embodiment has 

brackets are placed on the lancimL ^ ^""^^ °^ ^« 

automatically With no user mvolvementbased'ol ""'f""^''' """'"^^ 
type, tooth size, etc. P^^^^^"^ such as crown height, bracket 

As indicated above, the software al • 
virtual wu-e of any shape. %us as theT ^^'^ °^ 

foUow this shape, thereby indicating tontV ""'^^ 

feature allows the user to drive the tr 1 ^ '^^^^ ^^is 

-ire shape can be dictated by ^'^^ ^^^^ —-e shape. Conversely, the 

--entplancanbeonedriv:;;:::^^^^^^^^ ^ 
m the shape and design of the wire. ^^^ously, wide variety is possible 
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Furthermore, by providing for the simulation of teeth in both maxiUa and mandible 
together, the software provides for a wide variety in defining the maxillary/mandible 
relationship, the occlusal plane and the mid-line of the teeth. 

Another significant aspect of the software is that it provides for virtually unlimited 
5 3D measuring features using the marking icon and measurement feature described earlier. 
This feature offers a powerfiil diagnostic tool, as well as a tool for monitoring the progress 
of treatment and quantifying results. 

Because the teeth are represented as complete three-dimensional virtual objects, it is 
possible to detect the coUision between teeth or between teeth and brackets in the simulation 
10 of movement of teeth firom the current to the target situation. The point cloud representmg 
tooth objects defines a surface, when surfaces come in contact during the tooth movement a 
collision is simulated. This colhsion is detected by suitable collision detection algorithms. 
When the coUisions are detected, the user can be notified of the colUsion and resolve the 
conflict between teeth, for example by selecting one tooth as a primary tooth and movmg 
15 that tooth first in an initial stage of treatment, and then moving the other tooth m a 

lecondary sta^^ttelta^trirE"6i^feafi&esW " " 

application serial no. 09/451,609 filed November 30, 1999, now allowed, the contents of 
which are incorporated by reference herein. 

Another advantage of the instant treatment planning software is that it offer the user 
20 . real time simulation with immediate feedback on the effect of user specified tooth 
movement. When 'a value is entered into a field in the display or the user uses the 
navigalaoii tools, results are displayed immediately. Further the system offers arbitrary 
access to every object, including election, navigation and export 

25 A. Robot design 

Figure 88 is a schematic representation of a presently preferred archwire 
manufacturing system 34 shown in Figure 1. The key aspects of the system 34 shown in 
block diagram form in Figure 8SA. The system 34 includes a control system for 
controlling the operation of a wire bending robot 604. The control system in the Ulustrated 
30 embodiment comprises a general purpose computer 600 running a bending program and a 
robot controller 602. The computer 600 receives an input file firom the precision appUance 
service center conq)uter that contains information as to the location of bracket slots m three 
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L IT ~ ~ '^^^^O" --ponding to 

= device whete bends need to be .ade. and tbe Jbo. 
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Hence th. ♦ • ''^^ mounted to the table, 
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therebv h^A *i, , • ' *° ^ Position in space to 

tnereby bend the workpiece. The detail ti,« ^ - 
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"riDDi„, tool • . moveable ann 606. In an embodnnent in which the first 

snppmg tooUs mounted to the end of a moveable am, , ■ • 

moveaoie aim, the first gnppmg tool could retrieve 
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the next workpiece while the moveable arm places the finished worlspiece at an exit 
location. 

After an archwire is bent in accordance with an input file supplied to the computer 
600, the moveable gripping tool at the aid of the robot arm 606 places the wire (or 

5 workpiece being bent) at an exit location indicated at 616. A conveyor system 618 
including a plurality of trays 620 is provided for carrying the finished archwires wires 622 
from the exit location 616 to a labeling and packaging station 624. The labeling and 
packaging station 624 includes a printer 626 that prints a label for the archwire and a 
magazine 628 containing a supply of packages such as boxes or bags for the wires. A 

10 worker at the station 624 takes a label from the printer and applies it to the archwire 622 or 
to the package for the wire. The conveyor system 618 is also based on a commercially 
available, off-the-shelf conveyor system, such as of the type available from the Montech 
division of Montrac. 

The wire manufacturing system 34 includes a heating controller 612 responsive to 
15 commands and settings from the wire manufacturing computer 600. The confroUer 612 
controls the supply of current to heating elements 607A and 607B in the gripping fingers m 
the gripping tools in the robot, to thereby heat the gripping fingers above ambient 
temperature. Temperature sensors 605Aand 605B detect the temperature of the gripper 
fingers and are used for feedback confrol of the heating of the gripper fingers. A direct or 
20 indirect system for measuring the temperature of the workpiece may also be provided, such 
as infrared heat detector. The heating controller 612 also confrols a wire heating power 
supply 611 that supplies a current to the gripping.tools when tiiey are bending a wire. The 
powCT ^ply 611 is- used when the robot is bending shape memory materials or Titanium 
Molybdehum Alloys (TMA) materials, or possibly other materials. The current produces a 
25 resistive heating in the wire. The current is controlled via a wire heating current sensor 613 
so as to produce a wire temperature at which a bend formed in the wire is set into the 
material. The heating of the gripping fingers avoids excessive heat loss when resistive 
heating of the wire is performed. 

Figure 89 is a perspective view of a moveable robot arm 606 used in the 
30 manufacturing, system in Figure 88. In Figure 89, the gripping tool at the distal end 634 of 
the arm is omitted- In a preferred embodiment^ the moveable arm is based on an off-the- 
shelf six-axis robot arm and fixed tool. A suitable arm, fixed tool, and robot confroiler for 
the robot 604 is available from Staubli Unimation of Gennany. The Staubli robot arm and 
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fixed tool is custoxnized with oripoin. ^ PCT/usoi/n96. 
software, current heating subsyrtem fo "^'^ ^^^^-^ 

for bending archwires or other suitable Z^TZicT ^"^^^ " 
The arm 606 consists of a proximal T^'' 
5 of Figure 88 and a free distal end f^"-) • • ^® " which mounts to the table 610 
gripping tool 651 A of Figure 9^is moxmtT'^^^ ^ ^^^^ ^^^^ 
motion along six diiferL rotationaTls ^7^,^^ ^ ''''''''' 

Bumbered 1 - 6 in Figure 89 ITius th b ' ^"^'"^'"'^ "^^'^^ 

head portion 636 rotates relative to the L7Z ^ '° ''''' 

10 segment 638 about an axis indicated by arrow o ' c f " ''''' ^ 
about an axis indicated by aixow 3 by a joint 63Q T ^ 
about the axis indicated by the airow 4 A ' " " ^"""^ ^"^^ ^ segment 640 
rotates the tool flange 634 about the axis il7 
rotates the tool flange 634 about an axis indicaZT ^ 

FigurePOisperspectiveviewofrror^T'- 
Of the ann Joint 639 and the co.esponding motionT.I ^'^"^^ 
634. The motion commands for the robot ° ^ segment 640 and tool flange 

cable 650 which plugs into the base 63 0 of7e7^^ ^''^ ''''' ' 

Figure 91 is a detailed perspective vie^rf'^, . • 
20 mounted to the tool flange 634 at the d* al ^"^^^ g^PPi^g tool 651A that is 

Figuxe 90. Tt. const^ction shown in fILT, 1^^" ^ 

Figure 88 and Figure 92. The gripping to 16 *° ^^^"^ ^^^^"^ °^ 

fingers 652 and 654 which open anlel^irj^'^r'^'''' ' of opposed gripping 

orthodontic archwire 615 Apneumati '^^^^^ ^^^^^^^^^^ in this case an 
^^--axisrelat,Vetoflnger654,toL:i^^^^^^^^ 

releasea woxicpiecesuchasawire. Aposition. ^ 

of the fingers. Positxomng sensor 655 (Fig. 88A) detectsthe position 

^^^^P^^^tative embodiment, the archwires are «• 
such as Nitinol, a material based on Niclce, and Tita^^ 7 r ' ^'"^ 

> -terialscanbeheatedtoretaintheshapeofab^rf t '''"'"''"^^^^^ "^^^^ 
wire heating power supply 611 of Fi.n^e88A Z^"^'^"^*^™- Accordingly, the 
the fixed gripping tool and the movele ^ ' ' '™ ^^^^^ ^^^^^ 

creates a resistive heating of the wire sufficie^f™ aT^- T ^'^^^ 

ent tor the matenal to take a set according to the 
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shape of the wire as it is bent. To avoid dissipation of heat from the wire into the gripping 
fingers, the gripping JSngers 652 and 654 are preferably heated by electrical heating 
elements 607A and 607B. Conductors 660 and 662 supply current to the heating elements 
in the gripper fingers. 

5 The gripping tool 651A of Figure 91 further includes a force sensor 664 in the form 

of a strain gauge. The force sensor is designed to detect forces that the wire imparts to the 
gripping fingers after a bend has been placed in the wire or during the bending movement. 
The forces detected by the force sensor 664 are determined both in magnitude and in 
direction in three-dimensions. The use of the output from the force sensors to overbend 
1 0 wire is explained in fiarther detail below. 

Figure 92 is a perspective view of the fixed gripper 608 having a gripper tool 65 IB 
as shown in Figure 91 along with a moveable gripping tool 651 A located at the distal end 
632 of the moveable arm 606. An orthodontic archwire 615 is shown gripped by the 
gripping tools 651 A and 65 IB. The fixed gripper 60S and gripping tool 65 IB remain 
15 stationary relative to the base 610. The archwire is bent or twisted by the fixed gripping 
tool 651 grasping the archwire 615 and the moveable gripping tool 651 A also grasping the 
archwire 615, and then moveable gripping tooling 651 A bending the wire by moving to a 
new location in three-dimensional space relative to the fixed gripping tools. The location m 
space for the moveable arm to move to is determined by the input file fed to the robot 
20 computer 600. Basically, the input file consists of a series of point locations in a three 
dimensional coordinate system which correspond to bracket locations and orientation in a 
three-dimensional coordinate system for the arch, as described in more detail below. The 
manner of calculation of these points and generating movement connnands (i.e., arm 
position and switch states for the gripper fingers) for the robot's moveable arm and 
25 commands for the fixed gripper to bend the wire will be described in further detail below. 

Other possibilities exist for input files and calculation of the bending points. For 
example, in extraction cases, the wire is needed to close a gap between teeth and the wore 
serves as a guide or rail for the bracket to slide along to close the teeth. In this situation, a 
smooth curve is needed between the teeth to allow the brackets to slide the required amount. 
30 In this situation, the space between the teeth is divided into small sections, and wire 
coorduaates are obtained for each section. A series of small bends are formed at each 
section to generate the required smooth curve. It may be helpfiil in this situation to round 
the edges of the gripping fingers to help provide the desired smooth shape. As another 
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tools. For example, one part of the m' « S^PP'"^ 
^ Wires Of rectangul. cross-sectiorraiSr:^ aperies of reCanguUr contours to 
contours to grip round wires. ^^^'^^ circular 

a^i-hrg:s:;~s^^^^^^^ 

aUow the wire to slide thrnn<.hth ™^iece (e.g., wires). It may be desirable to 

- ^PPer exceed a worM^^^ 
i's grip on the woricpiece and aUows it ,0 sh^r " '""^ 



110 



wo 01/80761 PCT/USOl/11969 

Figure 93 is a perspective view of a magazine 6 14 of Figure 88 that holds a plurality 
of straight archwires needing to be bent in an presently preferred embodiment. Figure 94 is 
a detailed perspective view of the moveable gripping tool grasping one of the archwires 
from the magazine of Figure 93. 

5 The magazine 614 consists of a tray 670 having a set of parallel raised elements 672 

•that define a series of grooves 674 in the upper surface thereof The archwii-es 615 are 
placed in the grooves 674. The archwires are maintained spaced apart from each other m 
the tray. This permits the robot's moveable gripping tool 651 A to pick up a single archwire 
at a time from the magazine 614 as shown in Figure 94 and insert it into the fixed gripping 

10 tool to commence a bending operation. Also, the magazine 614 is positioned at a known 
location and the dimensions of the tray and slot features thereof are known precisely. This 
location, information is supplied to the robot control software and allows the gripping tool 
651 A to remove the archwires one at. a time from the magazine automatically and without 
human assistance. When the magazine 614 is empty a fiill one is placed at the same 

15 location. 

Figures 96 A and 96B are perspective views of alternative magazine constructions to 
the magazine of Figure 7. In Figure 96 A the magazine 6 14 consists of a cylindrical holder 
with a plurality of apertures 680 spaced from each other, each containing a straight archwure 
615. In Figure 96B, the archwires are in a rectangular holder with the apertures arranged in 
rows and columns. In either case, the moveable arm grips an individual one of the 
archwires and removes it from die magazine by virtue of the spacing of the wires from each 
other in die magazine and because die location of each wire in the magazme can be known- 
Figure 97 is a schematic illustration of a conveyor system 722 including a conveyor 
belt 724 that carries archwires 615 to the robot. Here, the robot is enclosed within a safety 
25 cage 726, A source feeds archwires 615 into slots 728 in the conveyor belt. When a wire 
has been bent and die robot is ready for a new wire, die belt advances one position and the 
robot grips die next wire placed on the belt 724. As anodier alternative, a spool of 
archwire can be fed to the robot and a cutting tool (not shown) provided for cutting the ware 
from die spool into a desired lengtii. The cutting tool could be incorporated mto the end of 
30 a second robot arm, for example. Still fiirther implanaitations are possible. 

It also possible for the archwire manufacturing system to have other workstations or 
workplaces in which one or more of the following tasks may be performed: loop bending. 
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B- Arcfawire Manufacture 

Tlic ^ * 

a representative embodiment of^T'^^''*^^ archwires (or other similar appliances) with. 
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5 and cutfng tte wire ■„ tagfl. 3, ^^T' ^ ^ 
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package at step 810. Packaged in a box or other 
The bending of the wire at step 806 • K 
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set the shape of the wire ^ ""^^^ bending motion, to 
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Rnhnt Tnout File 

The input ffle, which dictates the shape of an archwire after bending, will now be 
discussed in conjunction with Figures 99-106. The input file includes a set of matrices, one 
5 matrix for each bracket in the arch of the patient. Each matrix consists of a combination of 
a vector of location of a point on the bracket and a matrix of orientation, indicating the 
orientation of the bracket in three-dimensional space. Both the vector of location and the 
matrix of orientation are based on the position of the brackets on the teeth when the teeth are 
in a target situatioii. The target situation is developed by the orthodontist firom the scan of 
10 the dentition and the execution of a treatment planning using the treatment planning 
software at the clinic. 

Figure 99 illustrates the target situation for one arch 820 a patient. The target 
situation is a three dimensional virtual model of the teeth 822 in which virtual brackets 824 
are placed, for example, on the labial surface of the teeth. A coordinate system is defined 
15 for the arch 820 having an origin 826. The coordinate system is in three dimensions, with 
the X and Y dimensions lying in the plane of the arch and the Z direction pomting out of the 
page. The location of the origin 826 is not particularly important. In the illustrated 
embodiment, an average "mass" is assigned to each virtual tooth m the arch, and a center of 
"mass" is calculated for the arch 820 and the original 826 is located at that center. 
20 As shown in Figures 100 and 101, a vector of location 828 is defined for each 

bracket. The vector 828 extends firom the origin 826 to the center of the slot 830 m the 
bracket along the wall 832 of the bracket slot, i.e., to point 834. The vector of location 
consists of the X, Y and Z coordinates of the point 834 in the defined arch coordinate 
system. 

25 The orientation matrix consists of a 3X3 matrix of unit vectors of the form: 
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where X^ Xj and X3 are the X Y and Z components of the X unit vector shown m 
Figure 101, Y, Y. and Y3 are the X Y and Z components of the Y unit vector shown in 
Figure 101, and Zi Z, Z3 are the X, Y and Z components of the Z unit vector shown in 
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coordinates due to the known, location of the bracket slots and the known tangential and 
antitangential distances. The line segments can also be defined as a set of vectors having 
a location for the head of the vector and a magnitude and orientation in three directions. 
The foilo^ving discussion will use the set of points PI, P2, P3 . . . PN. In Figure 105, the 

5 slashes 842 indicate the end points of the bracket slot 830 of Figure 101. 

The bends need to be placed in the wire before point PI, between points P2 and P3, 
between points P4 and P5, etc., that is, between the bracket slots. The slot-to-slot bends of 
the complete archwire are bent "section by section. To fonn one slot-to-slot bend, the wire is 
fed so that the fixed gripper tool 65 IB and the robot arm gripper tool 651 A can grip the wire 

10 in its initial shape. The wire length between fixed gripper and robot arm gripper is equal to 
the curved length of the wire along the bend. The straight wire sections 840 between the 
bends have to fit to the bracket slots. To bend the wke into the demanded shape, the mam 
control computer 600 sends signals to the robot controller 602. The robot controller 602 
generates signals to move the robot aim 606 with the robot gripper tool 651 A into a new 

15 position. The movement path is defined by a bending trajectory. The bend is indicated at 
844 in Figure 106. 

To form one slot-to-slot bend (e.g., bend 844 between P2 and P3), there might be 
several of these bending movements necessary. One slot-to-slot bend is considered finished 
if two consecutive straight whe sections (e.g., between PI and P2 and between P3 and P4), 
20 have the desired relative positions between one another. 

To achieve this position, there are different approaches dependent on the wire 
material properties possible: a) bending material with elastic/plastic properties, such as 
stainless steel, b) bending material with sh^e memory properties, and c) beading TMA 
alloys. 

25 Material with elastic/plastic properties must be overbent to. compensate for the 

elastic part of the deformation. The overhead process, which is described in further detail 
below, can be defined as a closed loop control. Within the first bending step, the robot arm 
606 moves to a new position. Preferably the new position is equal to the planned position 
or to the planned position plus an amount of overbending. At the end of the move the forces 

30 and moments acting on the grippers are measured. They indicate the remaining elastic 
deformation in the wire. To determine the gripper position which correspond to the released 
wire shape, the robot arm 606 starts a new move in direction opposite to the actmg forces 
and moments. The forces correspond to a tiranslatibnal move, the moments to a rotational 
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Materials with shape memory properties and TMA will be bent to the planned 
position. To transfer this position into the memory of the alloy, the wire section between the 
two grippers is heated to a certain temperature for a certain time. The heating is possible e.g. 
by conductive resistance heating, laser, convection, radiation, or applying warm air or liquid 

5 to the material. Heating current and time must be appropriately adjusted to the respective 
alloy, the wire section length and the wire shape. To warm-up the wke, the wire heating can 
start ahready during the bending movement to the planned position. To avoid a heat sink 
effect at the gripper fingers and to ensure that the complete inter-bracket section of the wire 
obtains the necessary heating, the gripper fingers 652, 654 (Figure 91) or at least the contact 

10 areas of gripper and wire are heated too. The grippers may be heated continuously during 
the production process of the whole archwire. To compensate for an incomplete transition of 
the bending position to tiie alloy memory, there can be defined a certain amount of 
overbending. 

In bending TMA materials, the material can be heated to a high temperature where 
15 there is no springback, however when the material cools, it retains its springback properties. 
The procedure for bending such materials is as follows: 1) heat the gripper fingers; 2) bend 
the wire to the desired configuration; 3) heat the wire up to the temperature where the 
springback tendency no longer exists; 4) torn off the heat source and aUow the wire to cool, 
and 5) advance the wire to the next position for bending; and then repeat steps 1) -5). 
20 The bending of- the wure firom one section to the next requires that the location and 

aHgmnent of one straight wire section (i), for example P3 to P4, is defined in reference to 
the previous straight wire section (i-I) in the counting order defined in the bending system. 
The origin of the bending system is defined at the end of the straight wire section (i-l), 
which aims towards the foUowing straight section (i). The x-axis is equal to the direction of 
25 the straight wire section (i-l) directed to section (i). For wires with rectangular cross-section 
the y-axis is perpendicular to x and in direction to the wider dimension of the wire. For 
quadratic or circular cross-section the y-axis must be perpendicular to x and can be chosen 
according to practical reasons. The x,y,z-axis follow the right hand rule. 

The bracket slot data as described above needs to be transformed to bending data for 
30 use by the robot controller 602. This is done by calculation, fi»m the position of the 
bracket center point 834 (Figure 100) to the position of the straight wire section center point 
(located along the middle of the wure, point 840 in Figure 105). 



117 



wo 01/8076 J 



15 



0 



Next, there needs to be PCTWS(n/ivj69 

toga, of fl,e Wire Betwee. poi.,. p, ^ ^PP- co^ponding ,o *e 

be bent To mininn^ fte bending fece. and "'"^^ ^PP^ -i" 

r ~ — wbicbC^ 

ae ^pped .vire length *ouId be app„«J^ *° *^ =^™P-»< or to tte wire. 

*ape. If «.e iengtb is too *or, tbe ! '"^ 

too ,o,g, «.e wi,. Win ^ ^ ^ wiu be bigh ten«o«a ftrce.. 

The robot computer 600 the f 

» appropriate aigoritbn, ^^T'^"'" ^"""^ of the bent wire 

'^P^U fte Shape of.be wire nsin. nle" l""'^ ' » tbird order cnrve 
"SularspHne algoritbn. Idean/r:,^ using a 

i-usttated en-bodi^ent. a Be^er , ^ ^ me 

^alyScaldescriptronofas^oottcn^eldr " - 

lle lengtb Of d,e cnrve is obtai^^T »f 

<«=.-ions) along eacb segnaent of.be (in three 

» tte seg„,en.s can be set aAi^j„ and in "^^^ 
algorithm is a. fouows: *° -"'odimen. is 0.05 nm. The 



input: 

• Centerpointlocati 



and alignn^ent of two neighbored, 



as 4x4matrice with local vector i t ™ 

<« vector /, tangential vector / nnrmoi 
vector V) These matrices corre . " ^^^'^al 

iiinces correspond to the 4v4«, 

• '"^^ti^ and antitangenaal distances, and "^'^^^^'i^- 

• B=^er-distanceA,{en,piri,^,^^^ ' 

Tl.eBezierfonnnla,asln„wnbybW ■ . 

20X Ibe points of the sptt.e cnrve are ^'^ - abown in Figure 



118 



WO01/8(»761 PCT/USOl/11969 

Here it wiU be noted that the Bezier points Pj to P4 ia Figure 20A are not necessarily the 
points P.1-P4 of the wire segments shown in Figure 19, but rather are the points used to 
calculate theBezier spline length and shape. 
5 To describe the curved wire shape between the strai^t wire sections from slot 

(i - 1) to slot i , the Bezier points P„ P, , , P4 are calculated by: 

10 J.^l-^acj-l 

The wire length L and the N intermediate spline points can be calculated by flie algorithm 
shown in Figure 20B. 

The empirical value Bezier-distance must be set to make the calculated and 

15 actual bent wure shape tally. For orthodontic wires, a good assumption is 6^ = Ix . . . 2x the 

larger wire cross-section dimension. 

The bendmg trajectory needs to be calculated for each bend. The bending trajectory 
is a number of positions of the moveable arm's gripper 651A in relation to the fixed gripper 
65 IB, which comiect the start position and the destination position of a balding movement. 
20 In general there are translational and rotational movement components from each bendmg 
trajectory position to the next. 

F'or each bending trajectory position the calculated length of the curved wire must be 
equal to Ihe calculated length in the planned position. To avoid kinking condition for the 
wire the movement can be divided into two parts: 
25 I ■ Initial moveihent to steer the wire into a distinctly deformed shape. The initial 

movement is defined as a rotational transformation around an axis through the 
endpoint of the fixed gripper perpendicular, to the connecting vector of the start 
position and the end position. This is indicated in Figure 115A by the rotational 
motion of the moveable gripping tool 65 IB and movements a, b, c, and d. 
30 2. Finish movement to destination position. The finish movement is a gradual 
approach from the start position (or if there is an initial movement ficom the end 
position of the initial movement) to the destination position. The translational and 
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sensors 640 on the fixed and moveable gripping tools are used to deteraiine the position. 
This position is called the zero force position and corresponds to the released state of the 
wire. Forces, moments and the movements components are calculated in tiie main robot 
coordinate system of Figures 109 and 110. 
5 Depending on the nature of the material, some overbendmg of the wire may be 

needed. This would be indicated for example if the zero force position is not the same as 
the calculated position for the robot's moveable aim 606. To better understand the 
overbending principles, attention is directed to Figures 108 and 108. 

Figure 107 illustrates the relationship between force and deflection of wire. The 
10 solid Une 866 indicates how much force is needed to give a certain deflection of the wire. 
Where the force is less than Fl, when the force is released the wire returns to its initial state 
and experiences no deflection due to elastic properties of the wire. At force level Fl, some 
permanent deformation, i.e., plastic deformation, of the wire starts to take place. With a 
force level of F2, the wire is deflected an amount D2, but when the force is release from the 
15 wire the wire bends back to a deflection amount Dl, with the relaxation curve 868 
essentially parallel to the curve 866 up to force level Fl-, as shown. Thus, some level of 
force indicated at F3 is requured to be appUed to the wire such that when the force is 
removed the required deflection Ds is achieved in the wire. The feet that F3 is greater than 
F2 and that the deflection D3 is greater than D2 illustrates that some overbending is 
20 generally needed to yield the proper shape of the wire after the bending forces are removed. 

Figure 108 illustrates the stresses involved with wire material when it is bent The 
wu-e has one portion experiencing elongation stress, as indicated at 856, while compression 
stresses are experienced at locations 858, with the length of the arrows indicating the 
relative magnitude of the stress. With small bends, the forces at the center of the wire are 
25 small enough such that only elastic deformation occurs, while at the outside portion of the 
wue some plastic defomaation may occur. The result is that bending wire is a mixture oi 
elastic and plastic deformation, the character of which is determined by the amount of the 
bend, the wire material, and the cross-sectional sh£^e. 

To determine the amount of required overbending, there are several possibilities. 
30 One is a purely analytical solution Uke finite element analysis of wire. Alternatively, a 
piece of wire can be tested to determine its response to known forces, and the result stored 
as a caHbration table of bends. Basically, the curves in Figure 107 are obtamed 
experimentally. A more preferred approach uses force sensors 640 (Figure 8 8 A) on the 
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reference to the spatial relationship between the points where a bend needs to be placed m 
the wire (P3 and P2 in Figure 105, for example). 

At step 876, the force sensors are used to measure the residual forces imparted by 
the wire onto- the gripping tools, and if the forces are greater than some threshold, the 
5 moveable ^pping tool 65 1 A is moved to the position where the force measurement is zero 
or substantially zero. ■ .; 

Al' step 878 the actual position of the moveable gripping tool is measure using the 
position sensors- in the nioveable robot arm. 

, At step 880, a check is made to see if the difference between the actual position and 
10 the planned position is less than a limit. If not, new overbending values are calculated (step 
882), aiid a iiew bending curve is calculated to overbend the wire an additional amount, m 
the desired direction, to bring the wire closer to the desired shape (step 883). 

St^ps 874-883 are repeated until the difference between the actual position of tiie 
moveable gripping tool and the planned position is less than a limit. 
15 At step 884, the orror in the actual position relative to the plamed position is noted 

aad compensated for by correcting the next slot position. In particular, the next slot position 
represented by the next pair of points in the set of points in Figure 105 is translated m three 
dimensions by the amount of the error. This correction of the next slot position is needed so 
as to avoid propagation of an error in any given bend to all subsequent bends. 
20 At step 886, the overbending results from steps 874-882 are saved in memory and 

used in an ad^tive technique for estimating future overbending requirements as explained 
above. 

Figure 112 is a schematic representation of the overbending performed by the 
method of Figure 111: The line 900 represents the slot between the fingers of the fixed 
25 gripper and the line 902 represents the wire in various stages of bending. The dashed line 
904 indicates the movement of step 874, with Une 906 representing the slot of the moveable 
gripping tool between the gripping fingers where the wire is gripped in the planned position. 
The zero force position where the moveable gripper is moved to is indicated at 908 (step 
876 in Figure 111). There is both a rotational (dw) and translational (dx) aspect to the 
30 difference between the zero force position and the planned position. The last overbend 
position is shown as position 910 (the overbend calculated at st^ 870). The required 
correction is shown by the position 912. This new overbend position, calculated at step 880 
in Figure 1 1 1' is equal to the last overbend position 910 plus the planned position 906 minus 
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distance, as measured along the length of the wire, is maintained constant. The movement 
may perfonned in two stages (such as shown in Figures 1 15A and 1 15B). The result is that 
two bends 912 are placed in the wire, as shown in Figures 115C. Of course, a twist could 
also be perfonned in the wire if required by the prescription. 
5 After the bend has been placed in the wire, the steps shown in Figure 116A-116D 

are perfonned to advance the wire along to the position of the next bend. First, as indicated 
at Figure 116A, the moveable gripping tool 651 A is translated to the right an amount 
indicated by AX, This moves point P3 to the right of the fixed gripping tool by an amount 
AX, as shown in Figure 116B- Then, the moveable gripping tool releases the wire and re- 
10 grips the xme to the right of the fixed gripping tool as shown in Figure 116C and again 
translates to the right an amount sufficient to move point P4 to the right of the fixed 
gripping tool. The moveable gripping tool releases the wire again and grips the wire at 
point P4. The wire is again translated to the right such that the situation shown in Figure 
116D is obtained. The wire is now in position for a bend in Hie wire between P4 and P5. 
15 The process of Figure 1 1 1 and 1 12 occixrs again to calculate new bending and overbending 
positions and the bend is formed in the wire. -The process of Figure 1 16A-1 16D continues 
until all the bends have been formed in the archwire. When the final bend is complete, the 
wire is released from the moveable gripper at the exit location of the wire manufacturing 
system, and carried "by conveyor to the labeling and packaging station described earlier. 
20 Shape memory alloy materials require heating to take on the shape given by the bend 

produced in Figure 1 13E. Thus, for these wires, while the wire is held in the position shown 
by Figure 11 3E, heat is applied to the wire to raise the temperature of wire to the value 
needed to take the set. The temperature varies with the type of material. In the illustrated 
embodiment, a resistance heating system is used as described previously. The current is 
25 adjusted until the proper temperature is reached. The heat treating is deemed complete 
when the force sensors read zero (or less than some limit) when the wire is held by the 
grippers in the planned position. The amount of current and time applied to the vdre is 
again stored information that can be used for future heating of the same type of wire; 

For some softer shape memory materials, e.g., NiTi, the force sensor 640 (Figure 
30 88A) provided in the gripping tools must be very sensitive to detect the small forces 
involved. While shape memory materials may not require force sensors at all, they can be 
used to give information as to the effectiveness of the heating process. 
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™age processing algorithms are used t «°PP"S '°o'- Pattern recognition and 

oalculate its shape. More «.an one caml ^ iT 

" f iKcessary to image the wire 
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sufficiently to calculate twist in the wire. The effects of gravity would have to be 
compensated for in any position measuring system in which the wire is not held by the 
moveable gripping tool. 

Thus, in one possible embodiment the robot further comprises an optical sensor 
5 system such as a CCD camera detecting the shape of the orthodontic appliance after the 
bend in said orthodontic appUance has been made, such as by releasing the appliance from 
. the moveable gripping tool and allowing the appliance to take its natural position, and then 
using the optical system to detect the shape of the appliance. 

It is also possible to use both the force measuring systems and the optical system as 
10 a final check on the shape. The force sensor system (e.g., coupled to the fixed and/or 
moveable gripping tools) detects forces generated by the orthodontic appUance after the 
orthodontic appliance has been bent. The moveable arm is operated to move the orthodontic 
appliance to a zero-force position in which the forces detected by the force system are below 
a predeteraiined threshold. The optical sensor system detects the shape of the orthodontic 
15 appliance in the zero-force position. The position detected by the optical sensor system can 
be used as a feedback mechanism for furdier-wire bending if the zero-force position is not 
the intended or desired configuration of the appliance. An optional type of sensor system 
would be calipers that contact the workpiece and responsively provide position information 
as to the location (i.e., bend) formed in the workpiece. 
20 For stainless steel wires, tiiere is generally no need for heat treatment of the wire. It 

is simply bent into the desired position, with overbending performed as required. The 
shorter the distance between endpoints of a bend, the greater the deformation in tiie wire, 
therefore the greater the predictability in the deformation. With orthodontic arcliwires, the 
situation can occur where there is a relatively long distance between bracket slots 
25 (particularly in the region of the molars) and it can be difficult to obtain a stable bending 
result. A preferred solution here is to make this distance shorter by adding on soirie length 
to the tangential distance of one slot position and the antitangential distance of the next slot 
position, as shown in Figure 117. Here, point P2 is extended in space to point P2', and 
point P3 is brought closer to point P2 by moving it to point P3 ' . The required bending of 
30 the wire is now calculated relative to points P3' and P2'. The bend placed between P3' and 
P2' is now sufficiently short that it can be formed with enough control. 

In practice, it known that after an archwire has been fitted to the patient's brackets, 
the archwire imparts forces to move the teeth to the desired position. However, after a 
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fingers and associated structures may be optimized depending on the workpiece or 
appliance in question. However, the principles of operation are basically the same. 

For example, the robot of the present invention is particularly useful for bending 
fixation plates, rods, compression plates and the like, for example facial, cranial, spinal, 

5 hand, and long bone and other osteosynthesis plates, such as, for example, the titanium 
appliances provided by Leibinger Gmbh of Germany. These fixation plates may consists of, 
for example, an elongate skeletal firame having a plurality of apertures for receiving screws, 
arranged in straight lengths, C, Y, J H, T or other shape configurations, or a long cylindrical 
rod. At the present, these appliances are manually bent by tiie surgeon to the shape of the 

10 bone in the operating room using special manual bending tools. It is possible to automate 
this process and bend the plates in advance using the principles of the present invention. In 
particular, the shape of the bone or bone firagmaits is obtained a CAT scan, firom a scan of 
the exposed bone using a hand-held scanner (such as described above. Once a three- 
dimensional virtual model of the bone is obtained, e.g., from CAT scan data, the virtual 

15 model is manipulated using a computer to fiise the bones together in the desired position. 
The surgeon then overlays the titiree^dimensional virtual implant in the desired location on 
the virtual model, and bends the virtual implant using the user interface of a general purpose 
computer storing the virtual model of the bone and implant. The required shape of the 
implant to fit the bone in the desired location is derived. 

20 Alternatively, a physical model of the bone in the desired configuration can be 

manufactured ftom the virtual model using stereolithography (SLA), three-dimensional 
lithography, or other known technology, and the sh^e of the implant derived firom the 
physical model. 

As another alternative, a SLA physical model of the bones (e.g., skull) is made firom 
25 a CT scan or other source, and the surgeon performs a simulated surgery on the physical 
model to place the bones in the desired condition. The model is then scanned with an optical 
scanner and a virtual model of the bones in the desired condition is obtained, as described in 
the patent apphcation of Rudger Rubbert et aL, cited above. The virtual fixation device is 
then compared or fitted to the virtual model of the bones to arrive at a desired shape of the 
30 fixation device. 

In either situation, the shape of the implant is then translated to the robot controller 
as a series of straight sections and bends of known geometry (and specifically position 
commands for the moveable gripping tool relative to the fixed gripping tool). The 
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Orthodontic Appliance Ovtimization 

Generally, the present invention also provides for a method and apparatus for arch 
wire receptacle (e.g., brackets, bands, headgear tubes, etc.) optimization. Such a method 
and apparatus includes processing that begins by obtaining a digital model of an orthodontic 
structure of an orthodontic patient. This may be done in accordance with the teachings 
described above. The processing then continues by retrieving a digital model of an initial 
arch wire receptacle (e.g., a bracket, a band, a headgear tube, etc.) that was selected from a 
plurahty of digital models of arch wire receptacles. Generation of a digital model of an arch 
wire receptacle may be further perfonned as described above. The processing then continues 
by digitally placing the digital model of the initial arch wire receptacle on a given tooth of 
the digital model of the orthodontic structure to provide a digital arch wire receptacle 
placement. The process then proceeds by retrieving a digital model of an arch wire. For the 
given tooth, the processing continues by digitally modeling a force system on the tooth 
based on the digital model of the initial arch wire receptacle, the digital arch wire receptacle 
15 placement, and the digital model of the arch wire. Having done this, when the force system 
is not optimal, the force systemls modified to" achieve an optimal force" system. The force 
system includes the effects of bonding pad thickness, bonding agent properties, the arch 
wire, Ugation forces, and biological characteristics (e.g., growth, tooth interference, tooth 
anatomy). Each of these factors may be adjusted individually or in combination to optimize 
the force system. With such a method and apparatus, optimal force systems given 
orthodontic parameter constraints may be obtained without the limitations of prior art 
orthodontic practices. 
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Determining and Monitoring O rthodontic Treatment 



Generally, the present invention also provides for a method and apparatus for 
treating an orthodontic patient. Such a method and apparatus include processing that begins 
by generating digital information reg^ding the orthodontic patient by a site orthodontic 
system. The digital information may include a three-dimensional scan inaage of the 
30 patient's orthodontic stmcture (e.g., teeth, jaw bone, gums, and other facial features), x-rays, 
clinical examination and measurements, dental history, medical history, photographs, etc. 
The site orthodontic system then transmits the digital information to an orthodontic server. 
The orthodontic server and/or the site orthodontic system creates an electronic patient 
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adjusted, the next step of the revised treatment plan is executed. This monitoring of a 
patient's progress and revising the treatment plan, when necessary, continues throughout the 
treatment. Thus, with such a method and apparatus, a scientific approach is provided to 
orthodontic treatment throughout the treatment, maiatains the treatment costs at reasonable 
levels, and provides a more consistent and reduced treatment time. In addition, the present 
method and ^paratus allow orthodontic practitioners to have just-in-time mventory, allows 
for treatment adjustments that would be required due to changes in tooth position and/or jaw 
development, and further allows for changes in the orthodontic structure if the force system 
becomes sub-optimal. 

Treating an Orthodontic Patient 



Generally, the present invention also provides for a method and apparatus for 
treating an orthodontic patient. Such a method and apparatus include processing that begins 
by generating digital information regarding the orthodontic patient by a site orthodontic 
15 system. The digital " information "may ' include a three-dimensional scan- image- of the 
patient's orthodontic structure (e.g., teeth, jaw bone, gums, and other fecial features), x-rays, 
clinical examination and measurements, dental history, medical history, photographs, etc. 
The site orthodontic system then transmits the digital information to an orthodontic server. 
The orthodontic server and/or the site orthodontic system creates an electronic patient 
20 record from the digital inforaiation. The electronic patient record includes, but is not 
Umited to, clinical examination interpretations, radiology examination measurements, 
automatic and manual cephalometric analysis, a created electronic cephalometric tracmgs, 
an electronic composite including an integration of the three-dimensional images, 
radiographic data (e.g., x-rays, CAT scans, MRI scans), photographs, a generated electromc 
25 articulation module, measurements and analysis of electronic models, data quality assurance 
checks, and/or supplemaital data. 

The orthodontic server then generates an initial treatment from the electronic patient 
record, wherein the initial freatment plan includes precise steps to obtain a desired 
orthodontic structure. In other words, the initial treatment plan includes a series of steps, 
30 wherem, for each step, a corresponding orthodontic apparatus will be designed to reshape 
the patient's orthodontic stracture into a desired structure for the precise step. The 
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Generally, the present invention also provides for a method and apparatus for 
producing a three-dimensional digital model of an orthodontic patient. Such a method and 
apparatus include processing that begins by obtaining data of an orthodontic structure of the 
orthodontic patient. The processing then continues by obtaining at least one scaling 
reference point of the orthodontic structure. For example, the scaling reference point may 
be a marking placed on a tooth prior to obtaining the data. The processing continues by 
scalmg the data of the orthodontic stricture based on the at least one seating reference pomt 
to produce scaled data. Note that the data may be video data, x-rays, CAT scans, 
ultrasound, and/or magnetic resonance images. The process then continues by obtaining at 
least two orientation reference points relating to the orthodontic stiiictiire (i.e., fixed 
physical attiibutes of the orthodontic structijre .that will not change, or wiU change with 
negligible effects, during the course of treatinent). The processing then continues by 
mapping the scaled data to a coordinate system based on at least two orientation reference 
points to produce an enhanced three-dimensional digital model of the orthodontic patient. 
1 5 With such a method and apparatus, the automated treatment of an orthodontic patient occurs 
in a scientific and controUed manner. By obtai nin g an accurate three-dimensional digit^ 
model that includes a dentition structure of the orthodontic patient, patient treatment can be 
simulated by a computer in three-dimensional space, thereby providing a scientific approach 
to orthodontic treatment. 
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Producing a Desired Tliree-Dimensional Digital Model of an Orthodontic Structure 



Generally, the present invention also provides for a method and apparatus for 
generating a three-dimensional digital model of a desired orthodontic stincture. Such a 

25 metiiod and apparatiis include processing that begins by obtaining a three-dimensional 
model of an actual orthodontic struchire, wherein the three-dimensional digital model is 
defined in x, y, z space. The processing then continues by generating an interim three- 
dimensional model of the desired orthodontic stinicture less teeth. The interim three- 
dimensional model is designed in x, y, z space and mcludes the deshed placement of an 

30 occlusal plane, an upper-arch foim, a lower-arch form, an upper-arch midline, and a lower- 
arch midline. The processing then contiuues by positioning the upper and lower teeth with 
respect to the interim digital model and tiie defined x, y, z space to obtain a first pass three- 
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begins by providing a Hst of health care services to a patient and/or care provider. The 
processing continues by prompting for input of digital information regarding the patient 
when a health care service has been selected. The processing continues by detemiining 
whether a sufficient amount of digital information has been received. If so, the processing 

5 continues by simulating treatment of a patient based on the digital information, a treatment 
objective, and normalized patient data. The processing then continues by generating the 
patient treatment plan in accordance with the simulating of the treatment when the 
simulated treatment results have been acknowledged. With such a method and apparatus, 
the generation of a patient treatment plan may be automated for particular types of health 

10 care services, including orthodontic care, dental care, and medical care. 

While presently preferred embodiments of the invention have been described for 
puiposes of illustration of the best mode contemplated by the inventors for practicing the 
mvention, wide variation from the details described herein is foreseen without departure 
1 5 from the spirit and scope of the invention. This true spirit and scope is to be determined by 
reference to the appended claims. The teim "bend",, as. used in the claims, is. interpreted to 
mean either a simple translation movement of the workpiece in one direction or a twist 
(rotation) of the woikpiece, unless the context clearly indicates otherwise. 
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1 . . CLAIMS 

An mteractive, orthodontic care system h...H 
a hand-held scanner and associir " 
cientition of the patient and processin 2°^"'^ ^"^^^^ 
dimensional model of the dentitions • ^ fcUj virtual, three- 

a data conditioning system processing the , 
responsiveiy generating a .ef • • " ^^ee-dimensional model and 

s a sec ot mdmdual, virfuaJ tu^^^ j- . 
representing teeth in the dentition of the patient- '^^^-^^^-^onal tooth objects 

a workstation comprising a user interface fordid, 
three-dimensional tooth objects and software adanted f ^^-^liyidual. virtual 

virtual three-dimensional tooth objects to ther b T """^"^^ ' "^"^ "'^P'^"*' 
three dimensions and parameters far . . ^ ^''^ ^ ^« Patient in 

ameters for a customized orthodontic appliance for said teeth. 

2- The system of claim 1 h * 

- -chmc. _ stalled in an orthodontic 

3- The system of cl'aim 1, wherein the hand-held 

m an orthodontic clinic, and wherein 'd da workstation are installed 

'^'"^ "f <=i™ 3, wherein , . 

5- The system of claim I forth 
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said appliance manufactmin^ facilitv '^'^^^ ^PPH^co manufacturing facility, 
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7. The system of claim 1, wherein . the. orthodontic appliance comprises set of 
orthodontic brackets and an archwire. 

. 8. The system of claim 1, wherein the orthodontic appliance comprises a retainer. 

9. The system of claim 1, wherein the orthodontic appUance comprises a substantially 
transparent aligning device. 

10. The system of claim 1, wherein the hand-held scanner is adapted for intra-oral 
scanning and comprises a Ught source, an electronic imaging device and a pattern source, 
said light source projecting a pattern onto said dentition, said electronic imaging device 
converting incident radiation into a two dimensional image of the pattern as reflected from 
said dentition. 

15 11. The system of claim 10, wherein the hand-held scanner further comprises a 
' " continuous ill\mination'Ughfsoufce-p^ dentition dunng. 

scaiming. 

12. The system of claim 1, wherein the hand-held scanner further comprises a video 
20 mode of operation in which a light source illuminates the dentition and an imaging device 

generates a video image of the dentition. 

13. The system of claim 12, wherem the scanner alternates in rapid succession between 
said video mode and a three-dimensional image capture mode. 

25 

14. The system of claim 1, further comprising a scanning workstation coupled to said 
hand-held scanner wherem said scanning workstation includes a voice-activated system m 
which operation of the scanner is controlled, at least in part, by voice commands. 

30 15. The system of claim 10, wherein said processing system associated with said hand- 
held scanner executes a registration ^gorithm on frames of the three-dimensional point 
clouds, each frame corresponding to an individual two-dimensional image captured by said 
electronic imaging device. 
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16. system of claim 15 h 

registration algorithm. ' ^ ^^g^stration algorithm comprises a cumulative 

^ 17. The system of claim 15 h 

held .ca^er executes a pmS'TT'"°°'^'^^''^'^°"''''' wia said hand- 

IS. ^"'--tiveoA„do„«ctreata™,p,a„^„„3,^^ 

a woifatatioa having ^ <a,„,^^ ^ oompn^: 

^a.-d processing executo a rontine ;°""''""^™«">i'; 
^ensional .com objects of .eea, J' ^! ^i^ua, «^ 

''^'""'""'^^^''"""''-locciusedsitaaaon,- ^ -"''el of fte 

Plamung and app J^^'g^°7^ J"**"" 

individual ttree-dimensional virtual tooth Z "^'"^ workstation may move the 

comprising a three-dimensional virtual nioll of^i!^" T^""' ^"'^ 

■ ~°munication equipment for ^ ■ . ^ ^ ° '"^'^ta^ion; and 
omesaidtee.hatades^LJl':^™-^^^^ 

19- ^"y««=^ofclaimI8.whereinsaidrt>utinefor, ' ■ 

dunensional .ood> models compares portions of said ZLZ^^' ""^""""^ ^ 

toftionto tbree^ensional virtual temptate teel ^ "'<'<^' "t^ 

20. The system of claim 18 whe * 

dentition is obtained from an intra-olrr ^-^'^ '^''"^^''^^^^ virtual model of the 
held scanner. ^^^^^ ^-tition of the patient with a hand- 

21- The system of claim 19, wherein the remote loc . 

comprises an orthodontic 
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22. The system, of claim 18, wherein the remote location comprises a precision appli 
manufacturing center. 



23. The system of claim 18, wherein the treatment planning software program fiirther 
5 comprises a feature for enabUng a user to interactively place a three-dimensional virtual 
orthodontic bracket onto the surface of an individual, virtual three-dimensional tooth model 
and adjust the position of said virtual bracket relative to said individual, virtual three- 
dimensional tooth model. 

10 24. The system of claim 23, wherein the precision appUance manufacturing center 
comprises a wire bending robot for customized shaping of an orthodontic archwire and a 
placement device for placement of orthodontic brackets at the location where the user 
placed the virtual bracket on the virtual three-dimensional tooth model. 

15 25. The system of claim 24, wherein said wire bending robot comprises a six-axis wure 
bending robot: . . 

26. The system of claim 24, wherein said placement device comprises a bracket 
placemait tray formed over a three-dimensional physical model of the dentition of the 

20 patient in a malocclusion having orthodontic brackets placed on the surface of the teeth at a 
location specified by the user. 

27. The system of claim 26, wherein the three-dimensional physical model is made from 
a rapid prototyping method selected from the group consisting of sterelithography. laser 

25 sintering, milling, and three-dimensional printing. 

28. An interactive, orthodontic care system for use at an orthodontic chnic treating a 
patient suffering from a malocclusion, comprising: 

a hand-held, intraoral scanner and associated processhig system for capturing 
30 images of the dentition of the patient and processing the unages to generate a full, virtual, 
three-dimensional model of the dentition in the malocclusion; 

a workstation having a display, a memory storing the three-dimensional virtual 
model of the dentition of a patient in a malocclused situation and a processing unit; 
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' "^^^^ a. a ..i^ o^,^:" ~ a 

coniniuiiications * ' 

10 orthodontic appliaoc=. ° ""'^ ■Mnufectming a customized 

2'- He system of claim 28 further 

g^ierating individual, virfi^i 'fl„^.JlTT ' ''"^'^'"^ = 

<^-- vi.^.ode,o.theti.tir:~^^ - -e- 

^ ^ malocclused situation,- 

The system of claim 29 
ofsaidwoi,ad„^- '"■''-""■PH^^- the prScesSizig« 

31- system of claim 70 • . 

33. The system of claim 78 • , 

- -----^en.ofo.;.ri::r:::r^- - - - 

34. The; 



system of claim 28, wheremfh u 

. whet.^ the orthodontic appliance con^,^ , ^„ 

35. The system of claim os . , 

30 aagningdevice. ' ""^''-^ ""'"'^-^^ ^PH-e con^^ . 

36. The system of claim i, • 

electronic imaging dev,V. ' ^"^^^^^^ ^^^^^ includes a liah. 

g^g device and a pattern source said liah. ^ ^ 

> said light som-ce projecting a pattena onto 
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said dentition, said electronic imaging device converting incident 'v-tiiation into a two 
dimensional image of the pattern as reflected from said dentition. 

37. The system of claim 36, wherein the hand-held scanner further comprises a 
5 continuous illumination Ught source providing general illumiriation. of the dentition during 

scaiming. 

38. The system of claim 28, wherein the scanner further comprises a video mode of 
operation in which a Ught source illuminates the dentition and an imaging device generates 

10 a video image of the dentition. 

39. The system of claim 38, wherein the scanner alternates in r^id succession between 
said video mode and a three-dimensional image capture mode. 

15 40. The system of claim 38, further comprising a scanning workstation coupled to said 
- hand-heldscanner wherein-said scanning workstation includes a voiceractiyated syst^^^^ 

which operation of the scanner is controlled, at least in part, by voice commands. 

41. The system of claim 36, wherein said processing system associated with said hand- 
20 held scanner executes a registration algorithm on frames of the three-dimensional point 

clouds, each frame corresponding to an individual two-dimensional image captured by said 
electronic imaging device. - - 

42. The system of claim 41 , wherein said registration algorithm comprises a cumulative 
25 registration algorithm. 

43. The system of claim 42, wherein said processing system associated with said hand- 
held scanner executes a pattern recognition routine for said two-dimensional images, a 
decodmg routine, and a three-dmiensional calculation routine generating a tiaree- 

30 dimensional cloud of points associated with each of said two-dimensional nnages. 



44. An orthodontic appliance manufacturing system, comprising: 
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a machine readable memory storing di<n,al „n„ 
vutual model of a malocclusion of . ^'^S a ftxee^menrional 

loca^on Of or^odontic b.clce. .„ be pirLl '"'^ '^'^ 

Storing digital data representing a tee. . "aJoccinsion. aaid memory further 

<he denn-«on and loca^on of orLotrrr""^ ^ °f *e padenrs .eeft and 

awi^bendingrobo,; = 



10 



an instrument for 



wift .he buckets Placed'.^l^l',!^;*""^"'^ '^"^'^ 
a forming device fomiing a biactef „il 

PMica,mode,of«,ema,occ.Jonwi«flt;Tr"'~**^~°^*^ 

wherein the bracket plac 

"-y fonned Witt spaces for pUc^^ T P^-^-a, model, said 

■ .cea. a. a desired loca^on 1 ^^^^^ ^ 

_ of oraodontic bmckets on tte_,nalocc JrT''' ^^""^ ^ ' 



20 45. The systom of claim 45 wh * • 

having a iixed grippfag and a ml^l^JT- " =^ '^^ 

™>- "--blegrrppmg ,ooi mormted to the end of a moveable 
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46. ^® system of claim 45 whe ' 

»odel comprises a stereoMog^aphr^^^^ ^ Wdimendonal 



47. The system of claim 45 ftofl, 

customised bracket bonding pad b^ed on t^eT'"' '^^^^'^^^ - 

orthodondc bracket on the snr&ce of a th. h °' " Wdimensioaal virtnal 

P^-t '^■*"=-"'°-'^-odelofthe dentition of a 

48. A system for manufacture of =. . • 

■-^^havingabase. comprising: Pa- ibr a bradcc, the 
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a memory storing a three-dimensional virtual model of the dentition and a three- 
dimensional virtual model of an orthodontic bracket- as . placed on the surface of a tooth m 
the three-dimensional virtual model of the dentition; 

a processing unit deriving from said three-dimensional virtual models the three- 
5 dimensional shape of a bracket bonding pad; and 

a machine system for manufacturing a bracket bonding pad according to said denved 

three-dimensional shape. 

49. A machine for bending a medical device into a desired configuration, comprismg 
10 a first gripping tool aud a second gripping tool, each of said gripping tools having a 

gripping structure for releasably holding said medical- device; 

a moveable arm having a firee distal end portion adapted to mount said second 
gripping tool to said moveable arm; 

said two moveable arm capable of independent movement relative to the first 
15 gripping tool about at least two different axes; 

. . a-control system operative of said of-the moveable arm-and the gripping tools to gnp 

the medical device while said gripping tools are separated firom each other and to cause the 
gripping tools to move relative to each other about at least two axes to thereby bend said 
medical device a desired amount. 

20 

50. The apparatus of claim 49, wherein said moveable arm is capable moving relative to 
said first gripping tool about three translation axes and three rotation axes. 

25 51. The apparatus of claim 49, wherein said apparatus further comprises a force sensor 
system for detecting forces generated by said medical device after said medical device has 
been bent by said first and second gripping tools. 

52. The apparatus of claim 49, wherein said force system comprises a first force sensor 
30 coupled to said first gripping tool and a second force sensor coupled to said second 
gripping tool. 



53. The apparatus of claim 49, wherein said medical device comprises an orthodontic 
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54. The apparatus ofclaim 49, wherein 



said medical device comprises a fixation plate. 
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5 55. The apparatus of claim 49, wherein said apparatus 

for appl^,,eat to saidmedical device whileTri^^^^^^ 

and second gripping tools. "^'^''^ ^'^^ 

!lpristg~ "'"^^ three-dimensional infonnation of an object, 

device Oriented along a second I IZ^^Io: " 
device fonning an image of said n T ^^^^^ 

-- relationship -for said-scanT,«- aiia z; coordmate system calibration 

coordtoaces associated wia iT Pa«em, said , pixel 

-..on . a. .eas. .02^^.2^ ^^'^ ^ ^ 

<iir«ctions, for said • ^ ^ u*nnaaoD in X and Y 

distance, and ~ ~ °' - - ieas. two diff^. Z 

.o.3a.r;::r^r:rs:::z;r^^^^"- 

dimension of points on said object r^n^^sT 
waging device. 

S saia projected pattern onto said electronic 

57. The scanner system of claim 56 h • 

infomiatlon in three dime.,,- ' '^"^ Processing system derives said spatial 

characteristics of lens syster" oP^cal 

systems associated with said nr«,-» *• 

taaging device, to a^erebycon^pensate for optical distZ """"" 

r opneal d,stortions present in said lens system. 

'^m>nlg^^Jl°'^^ a^e-iimensional surface information of an object. 
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positioning a scanner proximate to said object, said scanner having a projection 
system for projecting a pattern onto said object and an electronic imaging device generating 
two dimensional images of the reflection of said pattern off of said surface; 

moving said scanner and object relative to each other: 

capturing, with said electronic imaging device, a series of two dimensional images 
of said projection pattern after reflection of said pattem off of said object as said scanner 
and said surface move relative to each other; 

processing said series of images to obtain a series of frames of said object, each 
frame comprising spatial coordinates for a portion of the surface of said object in three 
dimensions, and 

registration overlapping areas of said frames to each other to generate a three- 
dimensional virtual model of said object, 

59. The method of claun 58, wherein said series of images are captured at a rate of at least 
one image per second and wherein said pattem is projected onto said object in a series of 

" flashes timed" at a rate in synchronism- with said imagexapt^ - - 

60. The method of claim 57,. wherein after initiation of said series of flashes said flashes 
continue automatically without human intervention. 

61 . The method of claim 57, wherein said step of processing further comprises a step of 
executing a decoding algorithm identifying specific portions of said projected pattem 
captured by said electronic imaging device after reflection from said otiject. * 

62. The method of claim 57, further comprising the step of storing a calibration 
relationship for said scanner wherein said processing system derives said spatial idformatiDn^ 
in three dimensions from said calibration relationship without use of optical characteristics 
of lens systems associated with said projection system and said electronic imaging device, 
to thereby compensate for optical distortions present in said lens, system. . 

63. The method of claim 62, wherein said calibration relationship comprises a 
mathematical function providing Z distance information and X and Y. values firom mput 
comprising pixel addresses in said electronic imaging device. 
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^4. The method of claim 6"^ h ■ • 

calibration table. -. - ™ calibration relationship comprises a 



5 65. The method of claim 57 fi,rth» 

modi™ storing two dtaendo^,' """"^^ °f providing a data storage 

to cove, said object ""'^^^ ~ ^'-^"^^ in,aging device needed 

65. 



ofsaid projected pattern in each of sa,d images 

.0 identic 

~ °^ pattern captnr^i by said electronic 

imaging device; and 

c) calculation of spatial coordinate., m .i, 

- -■--^-■o^-ddecoded.r:::.".'^"^^^^^'-!.,^..^^^^^^^ 

66. The method of claim 65 ii 

decoded points of said p,.iec.ed'J,:"™ ^ ^ ^"""""^^^ 
67. The method of claim 65 wh 

elec^onic imaging device is convertedTa T by said 

and Wherein said series offian.es ate stcrii " 

comprises the sten of e,eo,.K„ ■ " ^ whaein the method further 

-a^ee^LiirorrShr- 

acce.ih,e.os.dda.aptocesshrgsyste;,com;ri::U:ep":r'~^ 

ir:or:tr^r:r^ at .e^t t.o t.o. 

ject, wherem dunng scamiing the seamier and object are 
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moved relative to each other resulting in each image being taken from a different position 
relative to the surface of the object; 

(b) processing said data representing said set of images with said data processmg 
system so as to convert each of said two-dimensional images into a data representmg a 

5- frame and thereby generate a set of frames corresponding to said images, said set of frames 
comprising a cloud of individual points, each point in each frame expressed as a location m 
a three-dimensional coordinate system; 

(c) storing data representing said set of frames in said memory; and 

(d) fiirther processing said data representing said set of frames with said data 
10 processing system so as to register said frames relative to each other to thereby produce a 

three-dimensional virtual model of the object substantially consistent with all of said 
frames. 

69. The method of claim 68, wherem step (d) comprises the step of performmg a 
15 frame to frame registration of said set of frames, wherem each frame is registered with 

" respect'td"bnel)therframe in said" set of frames; - - _ 

70. The method of claim 68, wherein step (d) comprises the step of performing a 
cumulative regisfration of said set of frames, wherein at least some of said frames are 

20 registered to a pluraUty of other frames previously having been registered to other frames m 
said set of frames. 

71. The method of claim 68, wherein step (d) comprises perfomiing a frame to 
frame registration of said set of frames, wherein each frame is registered with respect to one . 

25 other frame in said set of frames, and thereafter performing a cumulative registration of said 
frames wherein at least some of said frames are registered to a pluraUty of other frames 
previously having been registered to other frames in said set of frames. 

72. The method of claim 68, wherein in step (d) one of said frames is a starting 
30 frame for registration, and wherein a spatial transformation relationship is derived for each 

of the other frames in said set of frames and stored in said memory, said spatial 
transformation relationship indicating how the points in said frame should be translated and 
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rotated in a three-dimensional coordinate system . • P^T/us«T/n969 
starting fiame. -^^ '^Sister said frames relative to said 

7^- The method of claim 7'^ • 

corresponds to the first 

"^^^ of claim 72 wh 

i-age taken of the object and in which othr^^ ''''' -n-esponds to selected 

vicinity of the object such that a substantial 1 ^^'^^ '"^^ °' ^ 
10 image and said other images. * "^"''^'^"P ^'^^^^ said selected 

Tie method of claim 68 h ■ 
registered to each other in a sequential h ' ''"^ ^^^^ are 

-.egreeofove^apincoveZ^lT'^^^^^^ 
15 ^ ^^^^'^oojectm said frames. 

^e_mettiod of claim 68 wh • 

scanning device and saidTbject is scann.w h «^°^Prises- a-hand%eld - 

saidobject. ""'''^"^^^-^^-d-I^eldscanningdevicaover 

- --^--,whereinsaidob,ectcomp.3esahuman. 
'^S. the method of claim 68 h ' 
anatomical structures. ' "^"^ ""^^^^^ comprises teeth and associated 

^9. The method of claim 78 h • 
anatomical sti-ucture of a hmnan. ' ^"^^"^ '^"^ ''^^^''^ comprises a model of an 

SO. The method of claim 68 h 
too a wo* station for said scazj^er. '^"^^^S ^t™ is Mcoiporaled 

awoj:::::^:~^°-°'<^^-cpa«.^ 

^vmgaprocessmg unit and a display; 
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a memory storing a virtual tbree-dimensional model of teeth and/or associated 
anatomical structures representing the dentition of a patient; and 

software executable by said processing unit to access said model and display said 
model on said display; , 

said software further including navigation tools enabUng a user to interactively 
manipulate said model on said display and simulate the movement of at least one tooth m 
said model relative to other teeth in said model. 

82. The apparatus of claim 81, fiirther comprising: 
a Hbraiy of virtual, three-dimensional orthodontic brackets, said software permitting 
a user to access said virtual brackets, place said virtual brackets on said virtual model of the 
dentition of the patient, and simultaneously display both said virtual brackets aud said . 
virtual model. 

83. The apparatus of claim 81, wherein said virtual model of teeth comprises a 
- set of virtual, individual three-dimensional tooth objects and- wherein individual- ones_of said 
virtual tooth objects can be mdividually selected and moved relative to other teeth m said 
set of virtual tooth objects. 

84. The apparatus of claim 81, wherein said virtual model comprises a set of 
virtual, individual three-dimensional tooth objects and wherein said software enables a user 
of said workstation to select an archform for said virtual model, wherein said set of tooth 
objects are moved with respect to said archform. 

85. The apparams of claim 81, wherein said virtual model comprises a set of 
virtual, individual three-dimensional tooth objects and wherein said set of tooth objects are 
moved with respect to each other to define an archform representing, at least m part, a 
proposed treatment for the patient. 

3 86. The apparatus of claim 84 or claim 85, wherein said software displays a 

screen permitting a user to modify the position of said individual tooth objects with respect 
to said at least one of the foUowmg: said archform, associated anatomical structures, and 
other teeth. 
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A system for creatine an " d' ' 
representing a tooth foxmd in a dentitil *^^"-^^^^i°°al virtual tooth model 

coirespondtog ,„ a tooth. i„oIt««„, , te^pia,,!^ ~ """^ '-"'^^ 

a -a ..t^Luinr:™^*"""""*^ 

model of at least a part of the dentition- ^ three-dimensional 

wherein said data processing system Ar.h. 
Virtual three-dimensionalmodel of at leLap^ThT"^^^ -^-e processing said 
the template object and responsivelv d • • '''' '"'^ '^"^ °^ 

tooth model. ™^ '^"^ mdividual, three-dimensional, virtual 

SS- Th® system of claim 87 h 

^ specific characteristics of a tooth seleJteTZrr '"''^'^ ^^^^^^ 

occlusion characteristics, two-dimensional cross ser' ^^^"^ "^'"^ 

Jataess, . cross-section m an occlusal plane, roots and 
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89. ■ The system of claim 87 f. 
• fPnn«i of a mash of potapn surfaces. ' "^'"^ ^ surface 

^ ... - l^-^ll^^l^ ^ "^"^ ""J- a Cloud Of 

SI. . .;^ <»*odontio woAstatio,l comprising. 

a matbiy c<?mprismg data storage rerons storin . 
dimensional anatomical stmctures inc J- " ^presenting virtual ttoK- 

n.ode, repr^^^g „ individual tooth 7 " ■^--'^=-'■'-1 virtual toott. 

smtcn^s;, °' " i^'ated surrounding anatomical 

model/ i«=-ce ..c,uding a dispia, for displa^g individual v^ tooth 

interactive software for allowina a user • , 

respect to said surrounding anatomical sLctures. '"^ 
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92. The workstation of claim 91, wherein said individual virtual tooth model is 
generated from a scan of the dentition of a patient and a virtual three-dimensional template 
tooth. 

5 93. The workstation of claim 9L wherein said mdividual, three-dimensional 

virtual tooth model further comprises a root for said tooth, said root derived from a vktual, 
three-dimensional template tooth root. 

94, The workstation of claim 91, wherein said memory further comprises data 
10 storage regions storing a pluraUty of virtual, three-dimensional template orthodontic 

brackets. 

95. The workstation of claim 94, wherein said workstation further comprises 
software for permitting a user to place one of said virtual, three-dimensional template 

15 orthodontic brackets onto the surface of said individual, virtual three-dimensional tooth 
model and display said placed template bracket on said individual, virtual three-dimensional 
tooth model. 
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//. Overbending Concept Deformation Controlled 

( Start ^ 



calculate overfaendtng values u 
(translation, rctatian) j 



,87' 



calculats bending curve up to 
planned pcsMon + 
overbending values 



Move to planned position plus 
overbenctng values 



I calculate bending curve to ovetfaend posiUcn with 
I spfine^ength correction j 



Move to F/M = 0 
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calculate new overbending values 
(translation, rotation) 




new uvciu»"w ^.w- \ 

= last overoend pos, + (planned pos. - zeropos.) 
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